Sunday, May 31, 2009

Intranet usage: what really needs to be tracked?

A colleagues asked me what I would consider to be the most important factors of intranet usage to track, considering both the need to demonstrate the effectiveness and ROI of the intranet, and also to use tracking results to drive future intranet development.

My short answer is:
  1. To demonstrate so called 'ROI', it is always useful to show overall usage (e.g. no of hits, no of contributions). Coming from a library and information science background, information and lbrary professionals know that usage/hits does not mean 'users getting the help they need', however, many senior executive/board members do like to read the numbers, so give it to them.
  2. I find the tracking of search query useful, as I look at the search queries every month and ask myself 'if users are looking for something using these search queries, does our intranet give them what they want?'
  3. To drive continuous improve, I think intranet team need to really understand what users find useful (or not) as part of their day-to-day work. Look through the users' eyes (not the intranet designer's eyes). Ask users to talk about their work and how they got help (and see if the intranet is even mentioned!). Don't ask them a direct question such as 'how does the intranet help?' The intranet should come out naturally if it does help them to solve problems, get ideas, access information.
In ERM, myself and my team have been collecting stories from users on a regularly basis. They all going into a narrative database (thanks to David Snowden and Steve Bealing who help to set it up, having said that, a good excel file works as well). Listening and analysing the patterns from these stories tell us where we are lagging behind. I use these stories to inform our quarterly planning exercise, to identify areas of improvement and to challenge my own assumptions.

So here is my suggestion, give the quantitative figures to decision makers, and listen carefully to your users' voices if you want to drive continuous improvement beyond looking at usage figures.

3 comments:

Brad said...

Bonnie,

I don't think giving the executives the quantitative information is sufficient. I think that both the quantitative AND qualitative information should be provided. In a previous workoplace where I facilitated a number of communities of practice, I used to collect the quantitative data and the stories. I then put the data and the stories into a 1-2 page summary, using the stories (highlighting one or two specific business outcomes that I chose for impact). I found that I would then get asked about the "story" and then I was able to use that opening to give more positive stories that reinforced the value of the communities. Although hit rates and participation rates were still over(?)valued by the executives, now they had a better understanding of the business context and outcomes from the CoP's from the selected stories and subsequent follow-up.

Regards,
Brad

Bonnie Cheuk said...

Brad, I think your approach is pragmatic. I like it. My take from it is that you use qualitative stories to give meaning to faceless quantitative data. Thanks for your comment.

bradhinton said...

Bonnie,

Well, we pretty much agree on the importance of quantitative and qualitattive data. Your blog post initiated my thinking and my own subsequent blog post at my KM blog: http://bradhinton.wordpress.com