A couple weeks ago I attended the New Paradigms in Using Computers workshop at IBM Almaden. It’s always a small, friendly one-day gathering of Human-Computer Interaction researchers and practitioners, with invited talks from both academia and industry. This year’s focus was on the state of knowledge in our field: what we know about users, how we know it and how we learn it.
The CHI community has a good camaraderie, especially among the industry researchers. I suspect that’s because we’re all used to being the one designer, artist or sociologist surrounded by a company of computer scientists and engineers. Nothing brings together a professional community like commiseration, especially when it’s mixed with techniques for how to convince your management that what you do really is valuable to the company.
One of the interesting questions of the workshop was how to share knowledge within the interface-design community. Certainly we all benefit by sharing knowledge, standards and techniques, but for the industry researchers much of that information is a potential competitive advantage and therefore kept confidential. Especially here in Silicon Valley, that kind of institutional knowledge gets out into the community as a whole through employment churn, as researchers change labs throughout their careers.
Here are my notes from several of the talks. Standard disclaimers in place: these are just my notes of the event and subjected to my own filters and memory lapses. If you want the real story, get it from the respective horses’ mouths.
Mike Kuniavsky (Adaptive Path): Reverse the polarity
Kuniavsky’s first point was that the dot-com era brought new awareness of the need for usability testing and design. What was once an after-thought is now recognized as a primary driver for getting and keeping customers. He describes it as a crisis of purpose that has brought many new players together, including people involved in design, information science, HCI, ethnography and marketing. Techniques have also shifted from those that study single users (psychology, cognitive science) to those that study groups (marketing).
He then described the methodology used at his usability consulting company, which starts with an analysis of all stake-holder needs. These stake-holders include users of the system or Web site, as well as management, tech support, system maintainers, etc. Through this more holistic approach their solutions are more likely to have long-term benefit for their client because they can be more confident the right problem is being solved.
Colin Johnson (EyeTools): Understanding Users Through Their Eyes
EyeTools is a spin-off company from Stanford that provides user-testing of websites and software using their eye-tracking equipment. Their final reports show exactly where visitors are looking when they read a site, displayed either as they paths their eyes travel or as heat-map displays overlayed on top of the main page. They’ve shown some clients that not only are users not seeing the important marketing blurb on a page, but that they don’t even notice when the blurb is changed to complete gibberish.
One preliminary but interesting effect they’ve seen is that people have very different viewing patterns when asked to “say out loud what you’re looking at” than when they simply read naturally. If confirmed in a full study, that would imply the typical stream-of-consciousness commentary method of evaluating an interface may not get at natural usage patterns.
Bonnie John (HCI Institute, CMU): Data collection in support of US: Moving HCI to Level 5
Many Engineering fields have at least flirted with the Capability Maturity Model, which describes a process as being in one of five stages of maturity: Initial (chaotic), Repeatable, Defined, Managed, and Optimizing. The idea is to standardize the entire process such that methods are repeatable and testable, and the results of any change are predictable. Dr. John’s argument was that we, as a field, should strive towards levels 4 and 5.
Arguments against this idea came fast and furious. The first was that our work would become controlled by the “tyranny of the quantifiable.” Some research can not be easily quantified, but this does not make the research any less important. It was also pointed out that in HCI it’s very hard to tell what to measure, and doubly hard for politically-charged applications such as education. The compromise position was that especially critical niches might still benefit, such as usability of interfaces for NASA’s Martian lander. It was also pointed out that the field knows a great deal of rules about ergonomics, but such information has not yet been collected into a single volume.
Genevieve Bell (Intel): Lessons from the field: or, how not to do ethnography in 5 easy lessons
Dr. Bell had just gotten back from six months in Asia, studying daily interactions with technology in urban centers in India, Malaysia, Singapore, Indonesia, China and South Korea. The talk had lots of interesting insights about both Asian and Intel culture; here are a few tidbits:
- Asia is huge and extremely varied. This should come as a surprise to no one, but…
- …did come as a surprise to some at Intel.
- Religion is hugely important in some parts of Asia. For example, mobile phones in Malaysia and all around the Middle East have a much-loved feature: push a button and they point towards Mecca.
- In South Korea mobile phones are used to manage complex social networks to set up blind dates.
- Technology in Asia is used as a way to show your identity more than here in the US. Phones often are attached to stuffed animals or toys with which the owner identifies. She also showed cellphone covers from Malaysia with pictures of airplanes flying into the World Trade Center, and covers with Bin Laden’s picture and a quote from the Koran encouraging martyrdom. All hung out for sale in a normal retail store, wrapped in a Hello-Kitty plastic bag.
- In the US, mobile technology is still more predominantly information, with communication secondary. It is the reverse in Asia.
- Intel culture is such that people really want a big take-away message on a single PowerPoint slide. That’s how ideas have an impact there (and many other modern businesses I might add): have it on a slide that others can steal and put in their own talks. These days ideas are traded around like currency, three bullet-points per thought.
Merissa Mayer (Google): The science behind Google’s UI
Google has six basic steps to designing and testing their interfaces:
- Use existing bodies of research. For example, they mined all the psychology literature about what kind of fonts and color combinations were most readable.
- User-test existing products. Initial tests look at around 15 users to get feedback. Testing in early 2000, for example, revealed that users kept waiting “for the page to finish loading” because the layout is so sparse. This led to Google adding the copyright notice at the bottom, as an indicator that the page was complete.
- Internally test products in development. Ms. Mayer kept talking about feedback from “googlers,” clearly an extremely valuable internal testing resource. Eventually I asked what exactly googlers was, and whether it was some specific Q&A group. She laughed and said no, it was just the company-wide mailing list. Whenever they have a new interface or feature to test, they just send the URL to the entire company, and the next day find some 100+ emails with feedback. Google clearly has a fun and open company culture; it shows in everything they do. One nice quote: “Delightfulness has a value. People like to know the personality of the people behind a service.”
- Experiment on live site. Google will sign up sets of users (e.g. broadband users in a specific area) to try out a beta site. They used this technique to try out displaying thumbnails of resulting web pages. The feature wasn’t well-received: the thumbnails forced text down, so you could only fit four results above the fold instead of 5-7. Also, thumbnails were generally not useful unless you had seen the page before. However, they are trying the thumbnail experiment in Japan now, where usage is different. Due to differences in the keyboard, the Japanese use Google much more often to go to simple URLs like Yahoo!, rather than type the URL in directly.
- Analyze the logs. Google is constantly checking their logs to check things like how often the spell-checker feature is used.
- Lather, rinse, repeat. Iterate after each step. Google does a site-wide test once a month, and also tests other products like advertising products or the Google Toolbar.
As an interesting side-note, she says the purchase of Blogger wasn’t as strategic as it is seen to be outside of Google. They were purchased mainly because they were good engineers who needed resources, and it seemed to be a good match. There might be a move in the future to have them help with a service where blog owners can resell targeted ads on their site, should they want to monetize their publications.