In the s, when mainframes and punch cards were still in use, there were some people around the globe dealing with human-computer interaction. With the arrival of the personal computer, usability became more and more important. It was no longer just a small group of well-educated technicians and engineers who worked with computers, but an increasingly vast public. The first user tests took place back in — when the telephone keypad was developed at Bell Laboratories. And since the s, usability tests have been used increasingly frequently. With every new technical device we surround ourselves with, they become more important.
Virtual assistants, smartwear and the Internet of Things will not change that — on the contrary. Anyone looking for a really accurate definition of the terms should visit Wikipedia. Usability means that something can be used.
In other words: I can achieve my goal using the application effective. It takes as little effort as possible efficient. Crudely speaking, usability is simply what makes using something easy and convenient. The line between usability and user experience UX is another one of those things.
It exists in theory, but barely in practice. With UX we take it a step further: We also examine what the user did before they interacted with our product. And what they do afterwards. Nowadays, in a usability test, you usually also take into account aspects of UX. But most people still only talk about usability tests. User experience involves much more than usability. Usability tests involve setting aside your own opinion and seeing how other people handle an application. You can see which problems users have and what they understand straight away.
You might also see approaches or possible uses that surprise you. In any case, every user test takes you a step further. You move away from your gut feeling towards empirical bases for improving the application. Usability tests do have a large number of benefits — however, there are a few areas where you are better served by other methods. For example, when it comes to little graphic details or wording. These usually have such a minimal impact on usability that you would need huge numbers of participants to establish differences here.
In these tests, you can survey hundreds or thousands of users — an unrealistically high number for usability tests. So, for example, if there are only a few mock-ups, i. Complex menus, product filters or interactive maps can only be implemented in this way, however, if you put a great deal of time into them. It might be better in that case to create a focus group in which users discuss the designs. Or you can use another highly recommended method, the five-second test. This works as you might imagine: You show a user a screen for five seconds and ask them what they remember about it.
Which elements they have seen. This works remarkably well — and clearly shows which elements are the most striking.
- Cosmic Dawn: The Search for the First Stars and Galaxies (Astronomers Universe);
- Interact/Usability 1.
- See a Problem?.
- Moderating Usability Tests - 1st Edition.
And which get lost. After all, when you look at analytics, you notice that many pages of a website are only visited for two or three seconds before the users click on to somewhere else. A website or a screen of an application must make it clear very quickly what it offers, otherwise users are soon on their way again.
There are all kinds of best practices, but there are no ready-made templates you can simply copy and use for every test. The test objects, topics and things available to you are too different.
Moderating Usability Tests: Principles and Practices for Interacting
Ideally, you should begin with a good plan:. In the study concept or research plan, you set out the following points:. The study concept should be as clear and succinct as possible, and no more. Tips for communication within the team: Communicate insights. The classic is the test in the studio, in what is known as the uselab or usability lab. Here, you have good equipment, the entire infrastructure is available, and there is almost no room for nasty surprises.
But not everyone has their own usability lab.
An office where you will not be disturbed during the test also works. In that case, however, you need to build in enough time to set everything up and check before the start. The remote test is an alternative. In this test, your test subject sits at their computer and you sit at yours. You communicate online via Skype, Hangouts or another video chat tool where you can share your screen. You give instructions and observe — just like in a classical test in the lab.
These tests are also known as moderated remote tests, as you and your test subjects sit down at the same time. The alternatives are asynchronous tests, which are unmoderated. The test subjects receive a guide they work through themselves. Software records their interaction with the test object and you collect their comments and evaluations with a questionnaire. This saves you the work of moderation.
And it also takes a lot of time to evaluate the recordings and questionnaires. In my experience, unmoderated tests only really go much more quickly if you have a lot of test subjects — from around 20 subjects, there is a noticeable saving in terms of time. And in remote tests, some information is generally lost. I personally always prefer a test on site or in the uselab. This just means assessing usability. What do they mean and do you need them? Explorative formative I do an explorative or formative test early in the project. You want to know, for example, whether a classic navigation bar or a mega dropdown works better for your project.
Or how people generally handle configurators. In such case, you usually test prototypes — or competing applications. Assessment summative An assessment test or summative test is much closer to the finished product than a formative test. Here, you are testing prototypes, or usually even parts which are practically complete. The subjects can complete tasks independently, and as a moderator, you usually just need to guide and observe. Most tests we carry out on a day-to-day basis are summative tests. Evaluating validative Finally, the validation test or evaluation test is performed towards the end of the project.
In this test, you assess whether the problems you observed in previous tests have been resolved. And, above all, whether the product is working as it should. They give you an objective assessment of how you currently stand in comparison to earlier versions or the competition. The opposites of the very formal, scientifically accurate usability evaluation are the hallway test and guerilla test. The hallway is exactly where you will find the subjects for the test of the same name.
You simply grab a colleague who ideally has nothing to do with the project in question and has no specialist knowledge of the world of UX or the content of your project. The guerrilla test procedure is similar. These methods are cheap and affordable but have their own problems, due to the lack of choice of subjects close to your target audience and an environment which is difficult to control. A very important role that many people underestimate is that of the moderator, or test coordinator. As a moderator, you guide subjects through the test.
Your job is to make sure that they feel comfortable. That they act as natural as possible. That they perform the tasks and know what is expected of them in the test. The great skill here is in gently guiding the subjects but influencing them as little as possible. It sounds easy, but requires some experience before you can do it really well. As well as moderation, a second important task in usability testing is observation.
In many projects, the observer and reporter is a second person. That is good because you can then focus fully on the subjects as the moderator. One small disadvantage is that the subjects often feel a little more uncomfortable at the beginning — there are two people sitting next to them who are watching their fingers closely. If you are both the moderator and observer, then I strongly advise you to record the sessions on video. After all, in every single test, you will find yourself writing something down at the exact moment the test subject clicks on something you miss.
If you have a video recording, you can always take another look later and find out exactly what was going on.
ISBN 13: 9780123739339
As well as the moderator and potentially the observer, other people may attend the usability test. These are the stakeholders.
- Mediation Career Guide: A Strategic Approach to Building a Successful Practice;
- Smart Buildings. Advanced Materials and Nanotechnology to Improve Energy-Efficiency and Environmental Performance.
- Politics in the USA.
- Practical Ext JS Projects With Gears;
They are the colleagues who are working on the product and the decision-makers. The more of them are there, the better.
- Browse more videos?
- Introduction to Volume 10, Issue 1, November JUS!
- Psychology, spelling and education;
At the end, you can then do a workshop where you reflect on how you can solve the individual problems. But to ensure that the subjects are not unsettled by all the observers, and that the observers can talk during the test, the observers must sit in another room. A uselab will often have large two-way mirrors. The observers in the adjoining room can see the subjects through these, while the subjects can only see a large mirror. Alternatively, the image of the test object and the image of the subject and moderator is transmitted into the observation room by video.
The observers must not, however, appear in the test room. Only the subject, the moderator and potentially the observer taking notes should be in the test room. Stakeholders have no place there. The presence of several people may in itself be disruptive. The subject may also think that these people have something to do with the product and will be reserved.
And ultimately, those involved in the project find it difficult to stay calm and quiet and refrain from sniffing, laughing and, above all, asking questions during the user test.
IM MULTIMEDIA PROJECT MANAGEMENT
Many colleagues have a knee-jerk response to the question of when you should test: Test early, test often! And, importantly: The sooner you discover a problem, the less effort it takes to deal with it. If it only becomes apparent once the application has been created, designed and programmed, you have to go back many steps to deal with it. As an example, the tests I would ideally conduct when relaunching a big website or a new app:.
We also see what works well and which sections and functions we should keep. This allows a general valid navigation to be developed, for instance. However, most usability specialists still learn how to conduct tests through an apprentice system with little formal training. This book is the resource for new and experienced moderators to learn about the rules and practices for interacting.
Authors Dumas and Loring draw on their combined 40 years of usability testing experience to develop and present the most effective principles and practices - both practical and ethical - for moderating successful usability tests. The videos are available from the publisher's companion web site. Product details Format Paperback pages Dimensions x x Other books in this series.
Visual Thinking Colin Ware. Add to basket. Forms that Work Caroline Jarrett. Paper Prototyping Carolyn Snyder. GUI Bloopers 2. Moderating Usability Tests Joseph S. The Mobile Connection Rich Ling. Effective Prototyping with Excel Nevin Berger. Text Entry Systems I. Mobile Technology for Children Allison Druin. The goals of the test and your relationship with developers determine why and when to interact 2. Respect the participants? You have a responsibility to future users 4. The participants are the experts; you are in charge 5. Let the participants speak 7.
Your intuition can hurt and help you 8. Be unbiased 9. Don't give away information inadvertently Setting the tone Pre-test instructions Informed consent Chapter 6: Interacting During the Session Keeping the participant talking How much to interact Providing encouragement Dealing with stress Deciding when to give assistance How to move the participant along Chapter 7: Post-task activities What to do first Taking advantage of their knowledge Presenting ratings and questionnaires Making a good last impression Part III: Special topics Chapter 8: Interacting in a remote testing session Preparing for the session Getting the session started Interacting when you can't see the participant Chapter 9: In the Room vs.
Out of the Room Advantages and disadvantages of being in the room Advantages and disadvantages of being out of the room Chapter Summing Up Where do we go from here? Advice for novice administrators show more.
Review Text Joe and Beth really know their stuff, and they've put together a book that's enormously valuable for usability professionals and usability amateurs. Whether you've conducted hundreds of tests or are about to try your first one, you owe it to yourself--and your team Now you no longer have to worry about how to do that. Just follow Dumas and Loring's wonderful, practical advice and you will be prepared not only for typical encounters, but also for the unusual and unexpected, for doing remote testing, and for working with special populations.
Moderating Usability Tests is a great resource for anyone who interacts with usability test participants. Everyone talks about research methods, but the formal aspects of those methods only get you so far. The difference between getting a little data or a lot of data, only discovering problems or getting ideas about solutions, bias or validity, throw-away data versus generalizable insights, often depend on the soft skills, the ability to effectively moderate testing.
In the past, you were expected to get these skills through apprenticeships or trial and error. Moderating Usability Tests: Principles for Interacting with Participants removes the mystery and provides practical advice on how to get the most out of research. It will be invaluable to students learning about usability testing for the first time, people newly charged with evaluating products, and even old hands looking to refine and improve their technique.
In this generous book, Dumas and Loring give the benefit of their decades of experience and astute observation of both the foundational and the subtle aspects of conducting usability tests. Many questions you didn't think to ask until you were on the hot seat are answered here, and will help you achieve a level of confidence as a test moderator that may have seemed beyond reach, even if your participants are from challenging-to-test populations. With this highly ethical and thoroughly grounded program for developing moderator skills and avoiding pitfalls, Dumas and Loring make a strong contribution to the body of knowledge on testing products.
The big surprise of the book is that their clear, reasoned, and detailed suggestions about interacting with test participants and developers will likely spill over and improve your relationships with co-workers, family, neighbors, and friends. An unfortunate side effect of this awareness is that many people are conducting usability testing who have no idea how to do so in a way that will yield valid, reliable and useful data.
Other than the design of the test itself, proper and effective moderation of test sessions is one of the most important - and least understood - aspects of usability testing. Here is a book by two highly regarded experts that covers this topic thoroughly in a very readable format.