May 15, 2007 11:40 AM
How would you test the design of a book?
Not the content, but the quality of a book as a functioning information system?
It's a strange question—the book, after all, has been around for centuries, and its design is fairly stable. Conventions exist to guide what goes on the covers, the spine, the pages, and so on. Readers expect tables of contents, an index, chapters, sections, and pagination.
But it doesn't hurt to look for ways of carrying out those conventions more effectively. And maybe there is room for innovation when it comes to book design?
As a fledgling publisher, one focused on user experience design, Rosenfeld Media is trying to answer these questions. Our first manuscript, Indi Young's book on mental models, is going into production, and we're hoping to publish it late this summer. We've already designed the interiors—both print and PDF versions—for our initial series, based on a fair bit of market research (detailed here, here, and here). Now it's time to test those designs, and we could really use your help.
We've started crafting our test questions (listed below), but have a long way to go. It's especially unclear how to test for such qualities as readability and credibility. We're also sure that we've left out some important questions. Your suggestions would go a long way toward helping us provide you with better books.
Some background: we'll be using a print-on-demand service to create copies of the printed version for testing purposes, so our subjects will have access to a decent facsimile of the printed version as well as PDF. We'll likely test the design with 5-10 subjects who represent our eventual readership.
So how would you test the design of a book? Please feel free to comment below; we'd be grateful for your input.
Initial interview
Explore users' expectations regarding the design of books used for work.
User testing (done with printed and PDF versions of book)
Task analysis to determine if the design address common information needs, such as:
- Orientation (understanding what the book contains)
- Known-item searching (looking up information that you know is there)
- Open-ended finding (learning about a specific topic)
Post-test interview
Determine if:
- The design conveys overall credibility of the author and the publisher.
- The typography and layout support readability.
- The design met users' initial expectations.
Follow with questions regarding specific aspects of the design, graded on a Likert Scale.
Conclude with open-ended comments.
Comments
This may seem superfluous, but I know one thing I'd look for as a user of the book; something that could affect whether I used the book daily, weekly, etc.: will the book stay open by itself? Many of the books I use are read from begin to end, but also used as references as I'm working/writing at my computer.
Had I world enough and time, and wasn't needing to email clients, I would turn this into a larger comment on how reading is an activity--we tend to think of it as passive reception. But thinking of it from a very physical perspective and from a work-oriented perspective might help. For example, will the subjects have a desk in front of them? A computer too? I sometimes don't read without a computer in front of me to take notes. Speaking of notes, how easily does the paper take notes in the margins, using both pen and ink? Are there ample margins for notes? And so on.
Overall, I think this is a great idea: to take what's perhaps the most conventional information vehicles and try to, as Ezra Pound said, make it new.
Posted by: Joseph Tate | May 15, 2007 12:50 PM
Lou: I think this is fascinating. A couple of things that come to mind -- 1. Finding and discovering information in the book are of course super important, but are you doing anything contextual to see how that works in situ? Where will readers use the book-as-object? Sitting at a computer? Teaching in a classroom? Training in office meetings? Does the type leave enough room for underlining/note taking, or does the ink hold up ok to highlighting? How might its shape, binding, weight, etc lend itself to those activities?
2. I'm sure you've put a lot of thought into the cover and paper stock -- but man, that's such a selling point for me when I pick up a book in a store. I know people who have heard of Krug's book, gone to the store to buy it, picked it up and saw $40 and put it back. But another book that *felt* like $40 they spring for. Of course, durability is important too.
Random feedback, but there you go :-)
Posted by: Andrew Hinton | May 15, 2007 12:58 PM
Lou,
Before you actually look at what makes the book itself useful, one thing you might ask about is what makes a book findable. That is, are there important things, like a distinctive wrapper color, that enable you to find the book easily on a crowded desk or on a bookshelf?
Posted by: Fred Leise | May 15, 2007 02:02 PM
I'm kind of repeating what's above, but it seems like understanding context wold be the first big challenge. For example when use a technical book, the staying open part Mr. Tate suggests is key to me as well. And in the book store - yeah a feeling of quality and good design and a table of contents/back cover blurb that really show me how useful the book will be is key.
I'd be curious to read a study on how people do use books. I'm not going to use a book on experience design the same way I would one PHP. Unfortunately I'm likely to use read it in small chunks, in a catch-as-catch-can way the first time, then stick it on my shelf until I have the chance to apply something from it and grab it again. So small chunks, easy to find again are important to me. And I sound like an imbecile admitting this, but if the book doesn't have enough humor and narrative flow to it I might not make it through that first read through, and then I'll never know how good it could have been as a reference.
I have fond memories of using the polar bear book that way, as well as Eric Meyer's CSS book, Don't Make Me Think, The Elements of User Experience, Don Norman's Blueprint book. it's actually been a while since a book grabbed me in the same way, perhaps because this incessant blog reading has become my new method of professional development.
Posted by: Ben Brophy | May 15, 2007 02:26 PM
I, too, wonder about where the book will be used. Although we talk about reading the book on a coast-to-coast flight, I suspect this particular book will be open on people's desks or on a conference table quite a bit. It would be nice if it stayed open, but how do you do that without using a "cheap-looking" spiral binding?
Also, I would love to explore how the diagrams in the book might be connected to downloadable imagery or slides on the book site. Maybe introduce a certain icon that indicates that this image or bit of text is available on the site? A semi-translucent "bug" on the image itself? Being able to print out a larger copy of a diagram and stick it on the table might mitigate the book-staying-open binding problem somewhat. Of course, we would want to extend the available online material to some of the summary "how-to" text.
Posted by: Indi Young | May 15, 2007 02:37 PM
One very useful thing that I noticed in a book I read recently. On many pages information was cited by having a superscript number in the text which referenced a footnote, except they weren't at the bottom of the page, they were gathered into chapters at the back of the book.
At the top of each page of the notes section at the back of the book was a heading that told you that these were notes found on pages 117-120, for example. It was much easier to flip back and forth that way, rather than trying to match up note numbers and chapters...
I think the book was called 'Stumbling on Happiness' if you want to look.
Posted by: Deb Seys | May 15, 2007 02:51 PM
I just took a look at my bookshelf and pulled off a few that I've used and ones I return to to find out why. Here's how I use my books that are reference tools. And the reason I'm focusing on use as a reference tool, is that I think these books are more of an applied book - something we can use rather than just a read for fun.
1. Post it tabs on pages that have information I want to reference again in the future.
2. Margins for making notes
3. Pages that allow for highlighting that shows up well on that page, but doesn't bleed through easily
4. Support for full page images, text only pages, and pages with supporting images (great for providing educational content as well as visual examples)
5. Small enough to carry around (I love Designing Interactions by Bill Moggridge, but it's too big for my briefcase)
6. Visual hierarchy that allows me to scan the page quickly and easily (2-3 heading sizes and a very legible, yet forgiving font face)
7. A spine that's readable at a distance of 2-3 feet - I have a lot of books on my shelf. Some of the really small ones get lost in the suffle because I can't find them. That could be a danger for these 200pp and smaller books.
BTW, I know this is currently impossible in print, but I'd love to have a book system that allows me to look up a reference somewhere in the book and then not only brings me to the page, but the exact paragraph that includes that reference. Now that's useful. Maybe once we get to electronic paper RM can figure this out :).
Posted by: Todd Zaki Warfel | May 15, 2007 04:40 PM
I'd love to get a stream-of-consciousness as the reader goes through the book (this would work best for books that do get read or skimmed). Maybe by getting them to read through and put post-it notes on the page with whatever occurs to them as they go. It could be quite like a think-aloud usability test.
I think this would highlight when questions occur (even if they are answered later), parts of the book they skipped, whether illustrations helped the text, whether particular quotes really nailed a point, whether instructions were able to be followed etc etc.
Posted by: Donna Maurer | May 15, 2007 05:49 PM
How about the can-they-ride-the-subway-while-standing-and-turn-the-page-without-falling-over test?
Or, the does-it-fit-in-her-purse test?
And of course you must run the look-at-the-cover-and-know-how-to-find-a-sample-to-email-a-friend test.
Posted by: Victor Lombardi | May 15, 2007 06:06 PM
Big smiles here.
Wonderful suggestions, but most have to do with WHAT you'd like to see in our books. We've actually gathered a lot of this information in our past research, and our designs address many of your wonderful suggestions.
But HOW should we evaluate these designs? It's the testing approach that we're really looking for help wtih. :-)
Along those lines, my main take away is to look at context: how does the physical setting impact how the book will be used. Sounds like a job for Mr. Contextual Inquiry...
Posted by: Lou Rosenfeld | May 15, 2007 06:12 PM
I like the idea of sticking post-it notes on pages while reading a book. Another one is to give the reader a pencil and a marker and see what happens. If they are used very often then there are things to work on. Like when I mark many lines in the book then it means that text is not well organised. And I usually write questions near fragments that I don't understand or would like to check later. You can also test how long will it take a reader to finish a book.
And a good test of understanding is to tell someone to read a small chapter (few pages, not more) and then write a list of topics covered in it. I do it myself quite often when I need to learn something and then check if the list mathes the text. If I had missed something important then it's a clue that it had been written in a wrong way.
Posted by: Krzysztof Urbański | May 15, 2007 09:08 PM
Careful with attempting to measure stuff using a DIY Likert type scale. See
http://www.ucc.ie/hfrg/resources/qfaq1.html
for a somewhat didactic exposition of the dangers. Much more useful in this context would be some Critical Incident type survey. See
http://www.usabilitynet.org/tools/criticalincidents.htm
for a basic overview (not much more is needed!)
Posted by: Jurek Kirakowski | May 16, 2007 07:42 AM
It seems to me that you need to test for how effectively the book teaches the mental models to its readers. Perhaps your author could draw up a list of walk-away main points that she believes should be conveyed after a single reading of the book. If those points are not being conveyed effectively to test readers, it may indicate where you have problems and the opportunity for enhancing the text's *pedagogical* design features as well as others.
Good luck!
Posted by: Jennifer Roche | May 17, 2007 01:02 PM
Jennifer, great point. I'm hoping--maybe unrealistically--that we can test the book's design separately from its content. Maybe that won't be possible...
Posted by: Lou Rosenfeld | May 17, 2007 01:35 PM
Lou,
Credibility is going to be a hard one to test - but I guess, the way I might approach it is to ask the testee (towards the end of testing) how "credible" they find the book and the information within it on a scale. Next point out credibility boosting elements (e.g. publisher URL and information on the site, boilerplate information, references pages and bibliography etc.) of the design one at a time asking if each element adds credibility, removes credibility or has no impact on credibility at all. Then, having shown all the specific elements ask him/her if he/she would like to re-evalute the initial score given.
At the very least this will give you a first impression, a more considered score and some idea as to which elements are contributing to the overall impression of credibility or not.
Just a thought.
Posted by: Mark Thristan | May 21, 2007 09:39 AM
I'm hoping--maybe unrealistically--that we can test the book's design separately from its content. Maybe that won't be possible...
I don't see how that would even be possible. People don't normally notice book design unless it's either very bad or deliberately designed to be obtrusive. In a well designed book -- one where the design doesn't get in the way of conveying the information -- you simply don't see the design aspects.
By the same token, the design is going to be inseparable from the use for which the book is intended -- or rather, from how people will use it, which may or may not be the same thing. The content doesn't sound like it's meant to be a working technical manual -- it's meant to be read and thought about and applied, but not necessarily consulted in specific day-to-day tasks. Thus, designing it as if it were a technical manual may not be necessary.
Your Post-test Interview questions listed above may be problematic. If the book is well designed, everything but asking about the typography is likely to elicit a perfectly blank response, because general readers simply don't think about books that way. (The third one is especially problematic: what, precisely, do people actually expect from book design, when they think about it at all?)
...You're thinking about book pages as though they were website pages, aren't you?
Posted by: Wendall Sullivan | May 21, 2007 11:27 AM
At the risk of stating the berleeding obvious: reading a computer screen is quite different from reading a printed, bound book. If the PDF is simply a digital replica of the printed book's typeface, layout, page furniture etc, then I would use two overlapping sets of metrics for assessing the success of the design.
For example, in the usability testing I would expect to see search/find tasks completed faster with the PDF than with the printed book -- particularly if the PDF defaults to 'page plus bookmarks' as its default view and provides clickable links from ToC/index entries to relevant pages. However, these search/find aids don't necessarily make the PDF more usable than the printed book. Can't use what you can't find, and I know quite a few people who might save the PDF to their fileserver or local hard disk... and then never be able to find it again in the morass of other files and folders ;-)
It may be helpful to add some 'contextual enquiry' questions in the post-test interview to get some sense of how likely people are to prefer the PDF vs the printed version, and in what contexts they're more likely to use each format.
Posted by: Margaret L Ruwoldt | May 22, 2007 06:36 PM
Mark, great suggestion. At the very least, the a post-survey could ask for ratings for credibility and other qualities (borrowed from Peter Morville's UX honeycomb). We could also ask for explanations of extreme scores.
Hi Wendall, long time no see; hope you're doing well. I'm not convinced that we can't evaluate design; from my earlier research, readers--those in the UX community, at least--are very sensitive to book design issues. And I think it'd be a missed opportunity to reexamine book design--even if we simply end up validating existing conventions. But I do agree that asking about typography might be a bit pointless. And yes, overall we might not get too far in this pursuit, but I still feel like it'll be important to try.
Margaret: great points about PDF versus print. We've already design their two respective templates to support different uses: PDF will be more oriented toward quick reference, while print for sequential reading. BTW, as an information architect, I love your quote "Can't use what you can't find".
Posted by: Lou Rosenfeld | May 25, 2007 11:04 AM
Surely you'd start with what is the strategy for the book. Shat are you trying to achieve and who will read it, to what end?
Posted by: James Breeze | August 1, 2007 08:16 AM
Lou, as far as how to test the book, you should first start off with finding out how people are going to try and use the book. There's probably going to be some overlap across your books, but there will definitely be some uniqueness to them as well. For instance, I can see Luke's book being more of a reference book whereas Inidi's may be more educational (I've not seen either and am simply basing it on what I can decipher from the book sites - so, I could be totally wrong).
The use and goals of the consumer ultimately drive the scenarios you'll use to test the book. I'd be happy to help you design some tests. Maybe getting the initial draft into some hands of a few people first and interview them on usage will help answer some of these questions and frame the scenarios.
Once that's done, there's a technique called cultural probes that could work pretty well. Essentially, you send a packet out to the participants. That packet includes the book, a disposable camera, and a journal/survey booklet with instructions.
Participants are given instructions to journal their activities over a period of time. In this case reading the book. They jot down usage of the book, when, where, how, and make notes in the journal. The journal includes specific questions for them to answer as well as open ended comment areas. The disposable camera can be used to photograph the environment they're performing the tasks in and to collect other information. In this case, that might be photographing the book itself w/notes, tabs, etc. or other books they've got and how they've marked them up and used them.
Those are some initial thoughts.
Posted by: Todd Zaki Warfel | August 10, 2007 10:05 AM
I realize I come to this party very late, but I just discovered it.
For the questions you ask, one book you might want to have a look at is Colin Wheildon’s Type and Layout. Now, granted, he is writing primarily about advertising copy, but there are definite lessons to be learned about readability.
Beyond that, as a book designer, I allow myself to be guided by only one, general—if multi-part—“commandment: the purpose of a book’s design is to bring the writer’s work to the reader in a way that allows the reader to experience it in a way that is easy on the eyes; the design should be interesting enough, but not too interesting—it needs to hold onto the reader lightly, so that the writer’s words can grab the reader till their end, but should not be so evocative that it distracts the reader from the reading.
Stephen Tiano, Book Designer, Page Compositor & Layout Artist
steve@tianodesign.com
website: http://www.tianodesign.com
blog: http://www.tianodesign.com/blog
Posted by: Stephen Tiano | October 14, 2007 06:04 PM
Great advice; thanks Stephen.
The testing took place last Thursday; we're looking forward to discussing the results, but right now, we're hard at work applying them!
Posted by: Lou Rosenfeld | October 15, 2007 06:41 AM