I work for a funny little magazine. It’s online only, and published by a non-profit charitable foundation. Moreover, it purposefully gives all its content away for anyone to republish, for free – we care not whether someone reads it on our website or not, just that they read and engage with it at all, wherever they like. And it does go everywhere.
When your model is like this, what on earth does success look like? How do you measure that? When your content is everywhere, and the subsequent conversations too, how can you possibly know how well it’s doing or what people think?
Three years ago, this is what we at Mosaic wondered. So we came up with a system.
- Quantitative metrics: the straightforward stuff – unique pageviews, average time on page, referrals. All stuff you can get off of Google Analytics. We track our own website, and also have a republish tracker – building on an open source ‘tracking pixel’ that can be embedded into syndicators sites and which pings back data to us. But not everyone is able to put this in, so we also get what we can off of any known republishers who are kind enough (we ask nicely) to, confidentially of course, share their stats. We also use HotJar to generate heatmaps of our stories – a snapshot of how much and where people are engaging with our stories (i.e. how long they’re scrolling down and where they’re dropping off). Elsewhere, we track metrics from Facebook and YouTube insights, particularly for our videos, and in the last year have added podcast metrics – via our server Libsyn.
- Qualitative metrics: “Engagement” – comments, likes/reactions, shares, retweets, sentiment – sometimes seen as the softer stuff, but I actually think this is more important, particularly to us.
Yes, the quantitative stuff is important. Everybody likes to see numbers and graphs going up (especially higher ups). We watch those just as carefully as everyone else does. And we have to take them with a pinch of salt too. No web metrics are perfect, and it’s even harder to integrate when your data is coming from different systems and sources – how do we know what one publisher counts as a ‘view’ is the same as what another does, or if that’s the same as our data?
But given our publisher and mission statement, it was clear from the start that the qualitative stuff was going to be key.
When we launched in March 2014, we were – to be honest – taken aback at how quickly other publishers took to our Creative Commons offering. Though we’d done a lot of work to talk to potential partners about what we were doing to and to convince them of the quality and rigour of our work, we thought they’d be pretty cautious at first – fishing for more of a sense of exclusivity, seeing what the content was like first, and reader reactions to it. Instead we got a plethora of republishes right from day one including the BBC, Gizmodo, Digg, the Guardian and CNN.
With topics such as the safety of cycling and the legitimacy of female condoms, heated discussions started exploding all over the place – comments under Gizmodo and the Guardian, on forums such as Hacker News and Reddit, under other pages’ Facebook posts, and of course on Twitter – in reaction to our tweets and those of our republishers’ links.
It was a real buzz to see how big a response we were getting to our stories, and I got somewhat obsessed about reading and tracking every single thing – I lapped it up, I didn’t want to miss a thing, and was convinced that we needed to log it all lest we missed any of it – in the social media age, if you don’t capture it on the day it’s gone (or at least very very hard to find) tomorrow. We were just starting out and I sensed that we needed to demonstrate this impact to our managers and board to show just how viable the model was, and to hopefully keep their support – financially and in faith.
Initially, I favourited everything on Twitter I could find and then dumped it all into a massive Storify of each week (we publish a new longform story each week, a bit like launching a new book every week). I’d then add anything I saw on Facebook – on our page, but also if I saw any republishes, say, on Digg, I’d be sure to look up their posts on their page, then log every single (meaningful) comment – good or bad – from that. I’d even click on the ‘Shares’ under the post to see if any public shares had any meaningful comments alongside the share. I did the same for every forum and comments thread I could find across the web. For every one of our stories. I called all this ‘going down the rabbit hole’.
I dumped all of these into Storify but also started a spreadsheet on Google Docs to paste in each comment or tweet, links, dates, along with notes if a share, comment or retweet was from someone particularly ‘influential’ (e.g. noting the biog in their profile and follower count). This I felt was a good way of future-proofing the data and also making it meaningful, at a glance, for our Editor, the rest of the team, our Board and anyone else to see how we were doing, and what readers thought of our work, even if they only had a few minutes to glance it over.
I also started somewhat obsessively pinning every republish to our Pinterest boards – one for every story – a nice visual way of tracking republishes, while also maintaining a presence on a popular social network.
We still do this – it’s been extremely useful for everyone: writers, editors, my Editor when he’s writing a feedback report, us when we’re compiling Award entries. It’s also reasonably transparent – anyone can look at our Pinterest or Storify (the spreadsheet, for now, remains team eyes only).
I don’t do this all myself anymore. Frankly, I burned myself out trying – doing all that while running social media, production AND commissioning and editing the stories themselves is impossible. So after a year we took on a freelancer to help.
At the suggestion of a colleague, I tried Upwork, which allows you to recruit people to do web-based tasks all over the world. It’s mostly software developers and marketers – nobody did exactly what we were looking for, so I did a few searches and approached a few listees who seemed to have the skill in web marketing and keyword searches to handle the job. We found someone, and she’s great. Her work is invaluable to us and she’s a key part of our team.
Mosaic is now in its third year and this system of qualitative and quantitative tracking works out for us.
Going forward, I’d love to see some more investment (hint hint boss) in added resource for tracking, and to free up time particularly for analysis so we can really use the feedback and insights to improve our content. I’d love to afford a service like Chartbeat, that can integrate a lot of the quantitative metrics – website and social – into dashboards that can give you better at a glance insights that are actually meaningful (that said, I’ve reviewed a lot of services that people have pitched at us – lots of them are pants and nothing you can’t do if you can just take the time to look at the free metrics Google, Facebook and others give you).
I don’t think there is any shortcut to the qualitative tracking though. Especially when you have a content/publishing model like ours. If we were a commercial publisher, it may be a little easier, given that you would only have to track your own links and hence can tie that into a connected dashboard tracker like Chartbeat better.
Someone said to me recently that our way of doing things is a bit unusual and interesting, hence my motivation to share in this post. Hope it’s helpful to others in someway. If anyone needs me, I’ll be down the rabbit hole.
Mun Keat Looi 