Actionable Metrics – Something all publishers ought to pay attention to

Many sites are geared towards getting customers to download something and some attempts are made to get them to log in and personalize the site for themselves. As you build your solutions, its important to think in terms of what the user is getting out of all this and how they got to you site, what actions led them to download from your site. By paying attention to that you can increase repeat visits and downloads.

Recommend reading this article – Actionable Metrics – Say Hello to Cohort Analysis.

Here Ash Maurya talks about the difference between actionable metrics vs vanity metrics. The latter per Mr. Maurya  –

“only serve to document the current state of the product but offers no insight into how we got here or what to do next “

Couldn’t agree more.

So let me summarize some of the stuff he states because a) it helps me internalize these nuggets b) hopefully it gives anybody reading my post an overview. Definitely read his post!! And then I’ve tried to apply this to my own area of interest further below.

He offers three rules for actionable metrics –

1. Measure the right macro – Key things to know – How do users find you; Do users have a great first experience; Do they come back; How do you monetize; and Do they refer you.  Most importantly, don’t waste effort on simply driving signups when customer retention is already a problem. Focus on getting people a great first experience (getting what they want) and then getting them to come back.  This to me means, find out what your users are looking for, what they always download, make sure they can get it fast and at the point where they are converting, make their experience so good that they will want to do more or will come back for more. Sounds obvious but how often do we pay attention to what users really want and how good a first experience they are really having?

2. Create simple reports – We’re all impressed by the various charts and graphs and pages of information that many analytic tools produce or reports created on spreadsheets. But honestly, how many of us really digest every bit of information. I always save these off for later reading  but never get to them. Now compare that to a simple one page report (like the executive summary) or simple reports that speak to a specific problem; now that simplifies everything. I will definitely consume that immediately. So Mr. Maurya suggests looking at funnel reports based on Cohort analysis (or groups of people who share similar characteristics). For example a funnel report of users who are in different types of market segments or maybe different user types (guest vs logged in). Why do this? It tells you how any event or activity you did affects each cohort. Simple!

3. To make these metrics actionable, tie them back to specific user or users. This way you can find out who your activity worked for and who it didn’t. This enables you to contact these specific users for follow ups and feedback and thus get specific insights that you can act upon.

Feels so much like things we already do but the simplicity part is key here.

OK so let’s try an example specifically to the publishing industry. You have content that readers want to consume and they pay for it. More than likely users come from Google or some other search engine. When they arrive at your landing page, are they able to quickly consume the content. What is their first experience? Do they actually spend time consuming the information on your site or do they just download the PDF and go away? You probably want them to stay and read more. So focus on improving first experience – simple reading, fast page load, easy navigation, easy to understand functionality, etc. What do you measure when trying to improve the experience? User engagement perhaps. Additional page visits per session perhaps? Reuse of functionality? Revisits? Any thing else?

OK once you figure out what to measure, think about what kind of reports you want to see. Cohorts to me are by market segments – academic, corporate, government. Then there are guest users vs subscribers. What is in the funnel report here? #User visit page ->#User downloaded content->#user engaged with page->#user signed up for alerts perhaps (or something else) OR #user visited other pages.

Alright, once you have that and start getting reports, you then have to find specific users within each cohort that you can survey or email or talk to.

I think I got this. Any comments? Or other thoughts?


AB testing and pressing questions

Before you realize it, its been more than a year since i last posted. Its been mostly Twitter activity (@shiv17674) so its not like i’ve been completely holding back my thoughts.

Anyway, I find myself getting excited about my recent escapades with AB testing that i felt it would make for a nice post. Lately I’ve been experimenting with Optimizely a lot and its been eye opening to say the least. From small UI changes to complex functionality changes it sure looks like we have a come a long way since relying on simple user tests and surveys to tell us what a customer/user would prefer.

It all started after reading Avinash Kaushik‘s book Web Analytics 2.0 and the Always be Testing book (on Amazon). Shall we just say I have read about the top of the mountain and I’d like to go there. If you want a crash course on AB testing, I highly recommend this article from Smashing Magazine.

So when the opportunity presented itself, I jumped on it and now I’m like a kid in a candy store. (Do i go with Candy A or Candy B). Of course nerdy ecstasy aside, there are some really important organizational business decisions that will present themselves when we find the results of our experiments.

– Once we have the capability in place, everybody’s going to want to run these experiments. Organizational changes will have to be in place without overly burdening with red tape.  

– How do you prioritize which tests to run first?

– How do you make sure one test does not interfere with another?

– What if we can’t test everything? Will some changes remain untested?

– How long do we need to run a test and how does that influence our planning?

– I assume there are always things that just aren’t testable. Some things are just emotional and visual elements on a page. They are meant to make the customer comfortable within your product or add credibility. Whether to place those elements on the top or bottom or left or right will probably not be measurable. Or is there a way to measure that?

– What if the test reveals that your popular feature driving a sizeable chunk of your KPIs is actually hindering your customers ability to fully engage with your product?  

All these questions do have answers or at least I believe there are. But they’ll need to be evaluated and revised over time based on the experience we collect. I will try and post these thoughts on here when possible along with lessons learned. Certainly exciting times.

Scientific Publishing and Developer APIs

Rafael Sidi loves to quote Bob Dylan’s song ‘The times they are a-changing”… Certainly in the scientific publishing developer APIs world.

For years there were APIs available from the scientific abstracts databases Scopus, WoS, PubMed, etc.

Last year Elsevier launched SciVerse Application Marketplace that provides developers APIs (Application Programming Interface) to build applications that will integrate into the SciVerse suite of products (ScienceDirect, Scopus, Hub). Some examples –

  • SciverNote:  allows you to save ScienceDirect article abstracts and/or references to Evernote®
  • Top Reviews:  find the most relevant review articles in your Scopus search results
  • iSpeech Audio Reader: This one is pretty cool – reads your ScienceDirect articles aloud
Check out the current Applications Gallery.
You may also want to check out what Nature will be launching soon. This appears to be limited to Search APIs, scientific bookmarking tool (Connotea) APIs, and the social networking site (Nature Network) APIs. There’s a nice presentation (embedded below) put together by Chandran Honour that lays out the api landscape in scientific publishing (Oct 25, 2011)
PLoS has already announced their APIs for search and Article Metrics earlier this year. Springer has announced APIs to their Images database, APIs to the metadata for their journal/book content and API to query their OpenAccess content.
There are probably several that I haven’t mentioned here.
These are exciting times and it will be interesting to see if some understanding develops across publishers to standardize APIs. In the end the goal is to enable researchers to create solutions that are specific to their needs and hopefully help advance scientific research.

Elsevier Introduces Protein Interaction Browser

The Protein Interaction Browser allows readers to view and interactively explore protein-protein interaction PPI networks for all proteins in a scientific paper. The browser is currently available for the Elsevier-published journal FEBS Letters.

via Elsevier.

CrossRef Revises DOI Display Guidelines #yam

The new guidelines encourage CrossRef member publishers, affiliates, and others in the scholarly community to display CrossRef DOIs as full URLs in the online environment. (A CrossRef DOI is a persistent link to scholarly content.) To create a DOI URL, anyone can simply prepend to any DOI.

via CrossRef Blog: News Release: CrossRef Revises DOI Display Guidelines.

The Magical Number Seven

Very interesting paper that i just learned about

The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information

The Magical Number Seven.

What Is Your Mental Model? – Boxes and Arrows: The design behind the design

Old article but nice interview on UX… This quote caught my attention

Customers are just thinking about their reactions to the tool. They are not trying to squeeze water out of a water bottle, they are trying to quench their thirst. Of course you want to listen to them, but at the same time you want to interpret.

via What Is Your Mental Model? – Boxes and Arrows: The design behind the design.