• Home
  • Events
  • About
  • 10 Lessons Learned from Usability Testing

    March 19th, 2010

    This morning I sat through a great presentation from one of my fellow Canooeys about Usability Testing. The presentation and discussion were in German, so I have no idea what anybody said; but someone had brought Black Forest Cake to share. Any meeting with Black Forest Cake is a good meeting, without exception.

    I was introduced to Usability Testing way back in 2003 in the UST Graduate Program in Software. The class was taught by a great professor named Robin Kincaid whose distinguishing credential was that he was literally at Xerox Parc when Steve Jobs came to see the first GUI. Now, Prof. Kincaid wasn’t working on that machine in particular, he just happened to be standing in the hallway at the time, but that is a good enough credential for a Minnesota university. I was impressed enough to spend the next seven years carrying the mission of usability testing forward with varying degrees of success. Here is what I learned along the journey.

    You Can Afford It

    The low cost of discount or guerilla usability tests is the UX community’s worst kept secret. Where did everyone get the idea that you need a video camera and a dedicated usability lab? You need about 15 minutes of a real user’s time and maybe one hour per user of preparation: 4 hours total (for those not so quick at math: just recommended a sample of 4 users). The problem is trying to sell a manager “discount” usability and then trying to explain that discount doesn’t mean inferior. We’d be better off calling it Agile Usability so people identify usability with removing waste instead of vice versa.

    You Have the Tools

    PowerPoint or other presentation software makes a pretty good prototyping tool because you can quickly create mockups and even link a few slides together with hyperlinks. The user can then sit in front of an actual computer when exploring usability rather than having to deal with printed paper. But pen and paper do work well, and there’s even an entire book on Paper Prototyping. Balsamiq and other tools look great and produce impressive looking results, but require an initial investment to get started. If you need to impress a client then look at using Balsamiq. If you just want to make a better product then there is surely a zero cost way to start usability testing at your company by using the office supplies and software you already have.

    You Don’t Need Training

    You know what’s often better than being taught how to do something? Learning to do it yourself through practice. My last project included an initially reluctant designer. To kick start our usability practice, I ambushed him with his own mockups, and used him as the user in a testing session. Then we turned around and did the exact same thing with real users. Just showing someone how it is done within 30 minutes was better than several weeks of failed persuasion in meetings and retrospectives, and he turned into a great advocate for this style of testing. Over the course of the project we practiced and continually refined our own usability process ourselves, without training or consultants.

    You Don’t Need an Expert Opinion

    Usability testing means putting together a low cost prototype, showing it to real users, collecting the feedback, and then revising your prototype. An expert opinion is valuable when you have no access to your target market. For example, if you need accessibility features in your software, and you can’t find suitably impaired users to test with, then relying on an expert’s opinion is a good choice. But before calling in the consultants, ask yourself how these consultants became experts in the first place. The answer is no secret: they practiced usability testing and read some books. Instead of calling in the experts I recommend just becoming one yourself: practice, read, repeat.

    A Usability Session

    Usability Won’t Bloat Your Process

    So you’re already doing use cases, user stories, design documents, and copious amounts of meetings… can you really afford to add usability testing into your process? There is a legitimate fear that delaying development in order to produce paper mockups, testing scenarios, and user feedback will postpone final delivery even more. But take a hard look at your current process: aren’t the inputs to usability testing (mockups and a user scenario) already being produced? So called Software Design documents are oftentimes crammed full with screenshots and mockups, user stories and use cases are just testing scenarios with a different name, and many shops will spend hours on the design by committee merry-go-round debating how an interface should look. Usability doesn’t change the artifacts you produce, it just asks you to produce them at a time when they are useful instead of useless.

    You Do Have the Time

    Designing a user interface during a meeting is one of the most costly ways to create a UI. Eight people for one hour is a full eight hours of company time, and usually with a poor result. Instead, why not perform four hours of usability testing and then spend 15 minutes reviewing the results in a meeting with those same people. It takes the same amount of time in the end, so there is no additional cost to adopting this technique once. You only don’t have time if you insist on continuing to waste time in other ways.

    It Works Best in an Agile Cycle

    Here’s a process that has worked great for me in the past: with 2 week iterations the designer worked ahead one iteration, working with the users and testing his mockups. At the end of the iteration, we developers presented our working and finished software to the users, and the designer presented the system that was going to be produced in the next iteration. A dude writing some stuff on THE LENS! Whoa.Then development took the mockups into iteration planning where we used the artifacts as a key guide to producing estimates and work breakdowns, while QA used the mockups as input and guidance for test creation. We didn’t start with this process, but after a few iterations of tweaking we were all quite happy with what we ended up with. If you’re new to usability then use the by-the-book approach at first, but revise the process to fit your team and your company.

    It Is Not Testing

    The hallmark of a good test is that it passes or fails. It satisfies a requirement or it doesn’t. Usability tests, on the other hand, make things better. Skeptics of usability studies often use this lack of precise results to dismiss the whole endeavor, saying that results of testing cannot prove which is the best user interface. This is true. Design reviews also cannot prove the best software design. Both software and user interface design can help you decide the better option out of several alternatives, but both are limited by what alternatives are presented in the first place. If you are asked to choose between two crappy designs then your final design will be… guess what? Crappy. Don’t let the best be the enemy of the good. Many practices exist to provide an improvement, not perfection. This is one of them.

    You Don’t Need Usable Software (maybe)

    Everyone wants usable software, but not everyone needs it. Sometimes a complex, hard to learn interface is better than a simple, easy to use one. Always ask yourself how the software provides value. Frequently, the answer is that it doesn’t. If software is just a tool to getting transactions completed, then the more transactions completed the better off your company is. I’ve worked in several call centers and seen two drastically different approaches to usability. In a call center, all of the operators receive training, and time spent operating software is an expense. In this context a user interface should be optimized to be as fast as possible to complete transactions, even if it requires training to use. Several methods exist to measure the speed of a UI, including GOMS and Master Clerical Data (MCD). If your customer will financially benefit from a fast to operate but difficult to learn system, then stop worrying about what’s intuitive and start worrying about keystrokes and mouse movement. Some simple arithmetic can create an eye-opening return on investment proposal. For more on this idea I recommend reading Humane Interface by Jef Raskin.

    It is Surprisingly Hard to Sell

    If you aren’t doing usability testing, then someone, somewhere has the job of making user interface decisions. These decision makers have power, and few people will give up power for nothing in return. Maybe it is a developer who produces screenshots as part of a design process, or maybe it is a manager who debates and decides how a form should look. If you want to try usability testing then you need to convince these people that it is worthwhile. A developer is typically easy to convince, because you can replace an ad hoc UI design process with something more rigorous, appealing to their engineering principles and padding their CV. A manager or analyst is much harder to persuade because they often are not rewarded based on the quality of the finished product but instead based on the amount of their input that ends up in the finished product. Selling usability can oftentimes be interpreted as making part of their responsibilities obsolete. This is true, good, and a difficult pill to swallow. I recommend involving analysts as a key member of the usability team, so that the energy they had put into committee decisions can be refocused to a better process. As for selling managers on the idea… I don’t have any successful tactics and would love to hear feedback or experiences you might have.

    Today, after several years of practice, I am just as excited and hopeful for usability testing as I was when I started. Many, many software shops could benefit from low ceremony usability testing. The key to getting value from it is to keep investment and costs to a minimum. Train yourself, don’t commit to fancy tools, resist the allure of pricey consultants, and just get on with practicing. Happy Testing!

    Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
    • email
    • Print
    • Twitter
    • LinkedIn
    • XING
    • Facebook
    • Google Bookmarks

    Which JVM language is for you?

    March 18th, 2010

    Enjoy a new video of Canoo Fellow Dierk König in action:

    "Scala, Groovy, JRuby, Clojure - Which JVM language is for you?"

    "Scala, Groovy, JRuby, Clojure - Which JVM language is for you?"

    In this discussion, panel members Dierk König, Guillaume Laforge (Groovy), Charles Nutter (JRuby), Stefan Tilkov (Clojure) and Ted Neward (Scala) discuss with the audience the pros and cons of the popular JVM-based lanauges Scala, Groovy, JRuby and Clojure in order to attempt to reach a verdict of rank. The panellists try to logically wade through arguments based on the key concepts of each language along with their primary applications and try to resolve clichéd comparisons such as performance.

    Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
    • email
    • Print
    • Twitter
    • LinkedIn
    • XING
    • Facebook
    • Google Bookmarks

    css.php