This is the first in a series of posts exploring how we can get more good evidence into the hands of more people who can use it.
Marshall McLuhan famously said "The medium is the message". Maybe that was a bit of an exaggeration, but it sums up something I've been thinking about: What message does it send when someone presents evidence using a medium that's not so user or search-engine -friendly? Does it say "This isn't relevant to anyone but a few insiders, so it's OK if not many people see it."? Or maybe "People should have to dig for good evidence. It shouldn't be easy to find, interpret, and keep track of."? As expectations, publishing tools, and reading habits have changed, some conventional ways of working with evidence have fallen behind.
The skimming's the thing. There's new empirical evidence that when people read online, "read" usually means skimming. Jakob Nielsen, a web usability expert, writes that "On the average Web page, users have time to read at most 28% of the words during an average visit; 20% is more likely. We've known since our first studies of how users read on the Web that they typically don't read very much. Scanning text is an extremely common behavior for higher-literacy users; our recent eyetracking studies further validate this finding. The only thing we've been missing is a mathematical formula to quantify exactly how much (or how little) people read online. Now, thanks to new data, we have this as well." Nielsen is referring to a Feb 2008 ACM Transactions article, Not quite the average: An empirical study of Web use, which says "We found that Web browsing is a rapid activity even for pages with substantial content, which calls for page designs that allow for cursory reading." The excellent Read/Write Web blog covered this, saying "What Nielson found by analyzing the data in the study was that although people spend more time on pages with more words and more information, they only spend 4.4 seconds more for each additional 100 words. By calculating reading rates, he concluded that when you add more verbiage to a page, people will only read 18% of it."
This latest research only reinforces my belief that there's an urgent need to present evidence differently -- whether it's evidence offered in an engineering report that analyzes costs and benefits of a particular technology, or evidence presented by a B-school professor who has formally evaluated a popular management practice. I strongly believe that evidence-based management will have a much brighter future if we can improve on the traditional, unstructured, document-centric formats typically used when presenting research findings and other evidence.
What can we change? Here are some things I'd like to see happen, or happen to a greater degree (I'll be exploring these in future posts in this series):
Information design. Does the evidence pass the MEGO (My Eyes Glaze Over) test?
Small bites. Can we develop a more visual presentation format, particularly for abstracts and executive summaries? Possibly using some type of pattern language for evidence-based management?
Standardization. Perhaps we can adopt a scheme inspired by, but more flexible and far-reaching than, the levels of evidence applied in evidence-based medicine (where Level I represents a randomized, controlled trial, and so forth).
Openness. It's imperative that we bring more people into the conversation, making it easier for them to critique other people's evidence, and contribute their own in some manageable way. Some flavor of interactive media (or social media, if you prefer to call it that) can make that happen.
Markup & microformats. I'll be looking at how we might use state-of-the-art publishing technology and techniques -- such as microformats -- to structure information so search engines and web browsers can interpret it more effectively.
Comments