Usability and User-Centered Design at 3M:  An Interview with Kathryn Bohlke

by John Wooden, Director of Usability Services

Kathryn Bohlke is Manager of 3M IT’s User Information and Usability (UIU) Group. In this role, she has been instrumental in building a usability practice area, creating a comprehensive body of user interface design standards, and ensuring that applications and internal web sites at 3M are subjected to thorough UI design reviews. Kathy is also a Certified Usability Analyst through HFI and a graduate of the University of Minnesota’s Management of Technology Master’s program.

John Wooden, Director of Usability Services at Fredrickson Communications, spoke with Kathy about building and managing the UIU Group at 3M.

John:
During your time as 3M’s UIU Manager, has usability – as both a practice and an objective – become more widely accepted by 3M IT’s internal clients? Or do you still need to do a lot educating about the benefits of usability?

Kathy:
It really is a mixture regarding the acceptance of usability testing, but the awareness and use have increased for what I call “real” usability feedback. By that I mean conducting formal usability test sessions with actual end users as opposed to providing walk-throughs or training sessions and calling those usability reviews, or doing user acceptance testing with project team members and calling that usability testing.

However, the awareness is broader than the actual practice, because even though most agree that the solution should be usable, it is not always agreed on how to validate usability. Some stakeholders have different interpretations as to what constitutes usability testing. But overall the business side has been more open and accepting of usability testing than IT – maybe because they are closer to end users, market needs, and business value. IT has a history of building something, throwing it over the fence, and expecting users to work with it. Now users are pushing back – there is a greater expectation of usability – and this includes executive-level users as well. The increase in self-service applications may have led to this expectation of greater usability and productivity.

The biggest value add from usability within IT is having it part of our IT development methodology. This was partly a result of a Six Sigma project to identify the Cost of Poor Quality. Usability is considered a quality indicator, and so usability evaluation is a recurring activity throughout the overall IT methodology. To support this, various usability artifacts are provided, such as a Usability Testing Checklist, Usability Test Plan, Usability Test Script, and Usability Analysis Report. Usability evaluation is also part of multiple gate reviews. The project team holds a gate review at the end of each phase in the methodology to report progress and evaluate status to determine if the project should go into the next phase or not. Usability was not one of the attributes initially used to evaluate the status of the project, and it is now. So, if there is poor usability, the project has to decide whether or not to fix the issues, or to accept the risk.

So there’s definitely been progress, but we will never stop educating. All of our internal consulting engagements are an opportunity for communicating and educating about usability. We also give presentations at department team meetings, at Project Leader forums, and so on. There is a module for Usability in the IT Methodology’s training sessions, and I am referenced for consulting and services. Word-of-mouth has been good for educating and marketing as well. Our clients help spread the word too.

John:
You mentioned that not everyone agrees about how to validate usability. What are the most common misconceptions or misunderstandings about usability that you encounter when working with clients?

Kathy:
One point I sometimes have to emphasize is that I did not invent usability best practices – that there is a body of research supporting this field. Sometimes I have to validate usability best practices or guidelines by finding substantiating data from Jakob Nielsen or another published source. At times even that does not help because the person questioning my recommendation does not know who Jakob Nielsen is. But awareness has improved thanks to the internet. I can Google quickly to validate most things I recommend.

Another issue I have to explain is that user acceptance testing and training are not the same as usability testing. For example, I will meet with a project team and be told that they already did usability testing. I ask them to explain what method they used, which they do. I will then tell them that I call what they did training or user acceptance testing, not usability testing. When I explain real usability testing, I sometimes get a shocked reaction: “You mean, you aren’t going to teach them anything first?”

It can also be a problem when everyone on a project team thinks they are a usability expert. The truth is that you never know your end users as well as you think you do — so test with them. But even after testing, some people feel they still know better than what the user data indicates. One of the joys of testing is that you continue to learn. There is always another surprise result to react to and resolve.

Another important point is that user interface design is different from usability evaluation. Often I am asked to help a team with “usability,” and then I discover that they actually have a UI issue to resolve. The partnership between UI design and usability is tight, and some people don’t know how to distinguish between the two. What I need to do is make sure we are talking about the same thing before we get too far along into the discussion.

John:
How would you distinguish between the UI Designer role and Usability Analyst role?

Kathy:
I think UI design and usability analysis are different arts. And even within UI design, I make a distinction between transactional user interface design and graphical UI design. Transactional UI is getting the user from point A to point B efficiently and accurately — with little thinking. Graphical UI is making that path more visually interesting to cause the user to browse and contemplate. One type can blend in with the other, but to
have one person be excellent at both types of design is unusual. An analogy would be a writer who is equally good at documentation, training development, and copywriting.
I work with a variety of UI designer skill sets. The ones with a lot of programming experience design differently from the ones with more technical writing experience, yet both are UI designers — they just approach it differently. That is why we have documented UI standards and do usability testing. UI standards make all the designers agree on how a certain function or control is going to look and act. Then the usability testing makes the designers question their personal preferences and the documented standards and instead react to the user. Testing keeps designers honest.

Though having an advanced human factors degree, programming experience,
visual design experience, etc. can help people be better at the role, I think the true test at an interview for a UI Designer is to give them a piece of paper and pencil, and tell them to draw the landing page of an application that allows the user to order breakfast and lunch for a specific date and time. They are allowed to ask me five questions (hopefully used to clarify the requirements), and they need to be done in 30 minutes or less. That will separate the real designers from the wannabes. This exercise will also give you a clear indication if the person is a transactional or graphical UI designer. A programmer type may provide much more functionality than you want, a visual design type may want colored pencils, and a human factors type may want to know more about the user
personas.

The person I would hire will have a solid sketch with clear, easily visible controls that are easy to understand and that are aligned well, with everything clearly labeled and firmly anchored so I know where I am and what I can do next.

For usability testing, much the same holds true. The various educational experiences can help, but some important qualifications can’t necessarily be learned from a class. All facilitators have to be excellent listeners who don’t try to solve the problem before the user is done speaking, who don’t fixate on personal preferences, who don’t make assumptions based on the person’s appearance or speech pattern, who don’t over analyze data, etc. They need to stay focused and on track while looking under what they hear to figure out the root issues. For example, testers who can’t find the tabs at the top of the page may not mean you should make the tabs bigger, but instead may mean you need to change the background color so the tester can see the tabs better.

A really important qualification for both a UI Designer and Usability Specialist is that they are not “me” oriented. “I think” and “I like” are red flags to me. It is about the user — within the confines of tested standards and conventions.

John:
Usability and user interface design are now often included under the broader term “user experience” or UX. Different functional areas in companies have claimed UX as properly belonging in their area – for example, Marketing, IT, Communications. Do you think it belongs best in one particular functional area?

Kathy:
I think it belongs in an area that is not beholden to any other area. It
needs upper-level sponsorship and support so that it is required to be part of
each functional area’s DNA. My group is in IT, within a quality group. So the
group crosses over all areas of IT. The problem with being in IT is that it
can be perceived as an “IT thing” — not always good marketing. It would be
nice to have satellite groups or reps in multiple areas that all report in
to a central organization.

So when the marketing department, for example, has specific UI design challenges, those challenges could be addressed by their own UI/usability rep. But that rep would ultimately be beholden to the central organization. Organizations all have underlying political and social structures that have to be navigated, so the best fit needs to take those things into consideration as well.

John:
Based on lessons you’ve learned, what advice would you have for someone
trying to build a consistent usability practice in their organization?

Kathy:
First, help upper management “get it.” Usability needs a believer beyond the ones
who do it for a job. Second, track and market results. The down side for my
service is not having much follow up. Joining a project to do a test, provide the results and recommendations, and then going away is not effective. I have found that project teams still need assistance to create the correct UI based on the usability findings. We now follow up with the project to see what they have done and ask to be involved in meetings that prioritize the recommendations and decide how to resolve. We are trying to get more involvement with the project after the testing. This involvement will make it less difficult to spend the time creating a brag book or market via presentations. Third, provide templates, guidelines, standards, and stories to help people learn about it and try it on their own.

John:
There is a wealth of information here for organizations building a usability practice area. Thanks very much for sharing all the excellent insights Kathy.