I came across this story online about how Nielsen/NetRatings is going to drop its normal website rankings that use page views as a metric, and change to user session length instead. Much of this has to do with the advent of AJAX, with content loading on the same “page” and thus not being counted in the page views statistic Nielsen/NetRatings currently uses.
Forget how this impacts the current online leaders (the article says it will hurt Google, and in the same breath says it will help YouTube, go figure), I just found it interesting that they were still using page views as their main metric. While we certainly keep track of them here, we tend to put much more weight into user sessions. Take for example this comparison of page views vs. user sessions on our website, from Feb. ‘07:
tickets.walkerart.org
Page Views: 347,258
User Sessions: 2,581
blogs.walkerart.org
Page Views: 305,609
User Sessions: 105,387
Notice the difference? Our tickets website had more page views than our blogs did, but only 2% of the user sessions. Why? Well, it’s mainly because there are many more pages in our ticketing system to go through to place an order, as well as a few iframes here and there that just inflates the page count. By itself, one would think looking at the page views that tickets was the more popular site, but in reality many more people visit our blogs.
Like any statistic it’s important to look at multiple sets of data to come to a conclusion. With these two metrics we can not only find the depth of our visitors but also the breadth. Blog users don’t seem to dig as much, perhaps because they don’t have to compared to what’s required in a ticketing system checkout process, or perhaps because they haven’t found anything interesting to read and leave!
This is where Nielsen has made their change. Instead of just looking at simple page numbers (which is important to advertisers to count “impressions” of ads), they’re now wanting to see how long someone has spent on a website. This means the trend has changed from the number of impressions, to the overall impression length.
We also keep track of user session length on our websites. And again when combined with the other metrics, it adds another layer of info we can use to determine the relative success and weakness of our sites. Here are the user session lengths of the above sites for the same time period, in seconds:
tickets.walkerart.org
Session Length (secs): 589
blogs.walkerart.org
Session Length (secs): 281
Probably what you would expect. It took a lot longer for those users on tickets to weed through all those pages to place their order. However if you look at the ratio of users to page views for each site, and then look at the session length, you’ll notice that blog readers spend more time on each page during their sessions.
tickets.walkerart.org
Ave. Secs./Page: 4.4
blogs.walkerart.org
Ave. Secs./Page: 96.9
Obviously the tickets time is a bit skewed, because of iframes, robots and the like, but this shows that people do spend much more time on average on each page on our blog website than on tickets, even though the overall session length on blogs is less. This is good, it means people are getting through the ticketing software quickly, even with all the pages to load, and it also means people are actually staying on our blogs and (hopefully) reading.
Session length can also show us popular sites we may have otherwise missed. Take our Walker Channel from the same period:
channel.walkerart.org
Page Views: 21,596
User Sessions: 6,732
Session Length (secs): 574
Ave Secs/Page: 179
The user sessions on our channel aren’t super high, at least not in comparison to some of our other sites, and neither are the page views. However, the session length, and more importantly, the number of secs users spent per page is very high. Those people who do visit the Walker Channel like to spend a lot of time there. Perhaps this is something we should put more time into, to drive more users to this content? In fact, that’s exactly what we are starting to work on.
[…] Link to Article ajax Web Stats: Using Multiple Metrics » Posted at New Media Initiatives Blog […]
Pingback by University Update - AJAX - Web Stats: Using Multiple Metrics — 7/10/2007 @ 11:47 am
[…] The four main metrics we report are page views, unique visitors, user sessions, and user hours. We emphasize users sessions, believing they are the best comparison to the Center’s attendance numbers, while recognizing that all web statistics are subject to inherit caveats. For more on importance of using multiple metrics, see Brent’s recent post. […]
Pingback by New Media Initiatives Blog » Walker websites usage statistics — 7/13/2007 @ 3:40 pm
Brent, what do you think of this:
http://analog.cx/docs/webworks.html
in particular, the “What you can’t know” section? I’ve been using Analog forever and Stephen Turner may know more about web analysis than anyone. Basically, his point is that any measure of visits (user sessions) or visitors is a) inherently wrong, at least to a degree, and b) not comparable from site to site. So it makes sense to use page views as a point of comparison.
We tend to look at “page views by section,” eg to avoid the problem of comparing the Works of Art section (where people tend to click quickly through several pages) to the Visitor Info section (where we hope they find what they’re looking for without having to click very much at all). It’s like your tickets vs. blogs issue. And then we use “visits” internally, but mainly for two reasons:
1) as long as we don’t change our server-side caching, and there aren’t huge shifts in how users cache, we can track site growth over short (months to a year, say) periods of time.
2) The comparison to physical museum visits is too tempting to avoid making, even if I think it’s a little bogus. We try to back it up with surveys to figure out how often people come to website/museum (but that’s a lot of work).
“Page views” of course makes less sense with very dynamic pages, and I haven’t really figured out what to do about that. But it will mean that year-on-year comparisons are harder to make as we add more dynamic pages to the site.
Thanks,
Matt
Comment by Matt Morgan — 7/18/2007 @ 8:11 am
Matt, I think we’re pretty much on the same page here. I try to stress at the Walker that it’s not the numbers that really matter as much as the trends of the numbers. Turner is right on in saying that web stats are never perfectly accurate. Given the small amount of data in the request string of a log file, there really is no way they can be, as Turner explains well. But I think he is also hitting on worst case scenarios in his list. There is also a bit of self regulated min-maxing that goes on with stats. For every AOL user that counts as 10 visits on their own, there may be 10 people on one IP from a company counting as one person to offset it. Of course that is a big reach in itself, but I would venture a guess that good stats software would have a margin of error around 15% or so. That could be a lot but it’s enough to let us make some general observations at least.
On comparing site to site, I believe you can do this, as long as your metric tool remains the same. We use the same stats package (AWStats) across all of our sites, with the same settings, which means they’re all treated the same. That means even if our stats were are all wrong from real world numbers, all of the sites would be equally wrong. This is why I focus on the trends, and the comparisons between sites don’t focus on the numbers themselves but the disparity between them. Obviously people spend a lot more time on the Channel than on our Tickets site. That’s mainly what I want to know, the actual amount of time is less important to me.
Now between the Walker and the Met, since we use different stats packages, yes, it becomes much harder. Even if we both used Analog or AWStats we may have them configured differently. This is why I’m a big fan of Google Analytics. Mainly because it’s more accurate as it does use cookies and Javascript to capture data log files can’t, but also because everyone that uses it has the same settings, which makes comparison between sites easier (and when Google updates, everyone gets the update at the same time). The downside is of course plopping JS on every page on a site. Even for us that’s just not feasible at this time.
Anyway, I’m always interested in how others interpret their stats. It can be a bit of a guessing game, but it’s nice to get new ideas from others. Thanks for sharing.
Comment by Brent Gustafson — 7/18/2007 @ 9:27 am
[…] web host. For the most part, this has been acceptable. I’ve also read some posts that suggest using multiple trackers is valuable, and that page views, which I tend to focus on, are less valuable than user sessions, which my […]
Pingback by InsectaPod Blog » Analysis — 5/6/2008 @ 8:40 am