How Do We Improve the Learner Experience of LMS’s?

by John Wooden, Director of Usability Services

Corporate learning management systems do not appear very often in usability research literature. True, there’s been plenty of discussion of the issues that often arise during LMS implementations, especially as administrators try to integrate various types of content. (Fredrickson technical architect Monique Benson addressed these issues in an earlier article on our site.) But the end user (the learner rather than the administrator or instructor) experience of these applications has not been a high priority. This needs to change if the promise of these tools is ever to be realized. I don’t mean to suggest that vendors haven’t made improvements over the years, but more remains to be done.

As a client of ours explained in an email to me, “LMS’s are designed for learning professionals and not for mainstream learners. There’s a real need to make them more usable for the learners.” I agree completely. So what can be done about it?

The issue of LMS usability is complex, and there’s no quick five-step answer to the question in this article’s title. To mention just one knotty problem, it’s long been a concern that LMS’s are only searchable from within the system itself. This means that you can’t do a general intranet search and get LMS content in your search results. There are potential ways to address this, but they are challenging.

Still, it would be an important step in the right direction if more companies using these systems routinely conducted usability tests of their LMS’s with representative learners. When I’ve had the opportunity to do testing of major LMS’s at client sites, we’ve always gained insight into how people interact with these tools and how to improve the experience of using them.

Let’s take one LMS test project as an example of what you could do and what you might learn in your own organization.

Usability test questions

We began by drafting a usability test plan and recruiting about 15 testers in the client company to represent two user groups: (1) learners and (2) those who played a role in helping learners select and register for courses. Those in the latter group were often administrative assistants, rather than LMS administrators or instructors. In our test plan, we listed 10 key questions that we wanted to answer:

1. How do testers prefer to find courses – browsing the catalog, or using the search feature?
2. When do they browse versus search?
3. What tester performance issues arise when using the catalog and when using search?
4. Are the training content categories helpful and clear?
5. Do learners use the advanced search feature? What performance and preference issues arise? For example, do they know that they need to do an advanced search to find courses by location and type?
6. Are the search categories provided in advanced search helpful and clear?
7. Can users easily begin a new advanced search?
8. Are the icons meaningful to testers?
9. Can users easily register for a course?
10. Is it clear to testers how to check their transcript and change preferences?

We then drafted task scenarios designed to help us answer these questions. What follows is a high-level summary of the key issues the test uncovered.

Browse or search?

Even though the LMS user interface (UI) was designed for browsing, most testers ended up using the search feature. The learner’s starting point in the system was a topic index page that presented about 10 topic headings — such as Business Systems & Applications, Human Resources, Project Management, and Personal Effectiveness — in a very large bold font. When the learner clicked one of these headings, they would see the first page of an often lengthy course list. They could then click the course they wanted, read the description, and proceed to register for it.

Two main problems were evident when testers tried to find a course by browsing: (1) It wasn’t always obvious which category to check for a particular course. Although some headings were clear, others were ambiguous, such as Personal Effectiveness. (2) It took too much time to review the long lists of courses in some categories – for example, Project Management had 80 courses. So although the LMS interface invited the testers to browse by presenting big, bold category headings on the first page, this activity became too time consuming and difficult and was abandoned.

What testers did instead was to use the search, though this feature was much less prominent on the page than the topic categories. Unfortunately, the testers found that Search was not the easy alternative they were hoping for, because as some of them eventually realized to their frustration, the searches were scoped by default. In other words, they were limited to searching within just one topic category at a time. This meant that if they inadvertently looked for a course in the wrong category and saw no results, they might think that the course did not exist. (In a usability test of another LMS, we found the search logic was set at “Starts with” by default, leading to disastrous results for most testers who used it. Few testers knew precisely what the course names started with – “MS Project?” Sorry, no results. The user had to enter “Microsoft Project” to find anything.)

Advanced search

The advanced search feature of the LMS being tested was more than just a supplement for savvy users — it was a critical feature. It presented a number of additional search options that many learners would be interested in. For example, they could look for a classroom course in a particular physical location (the client company is based in many cities in different countries) or find learning material in a specific format (such as eLearning, CD, book, etc.). But in most cases the testers did not even see the Advanced Search option, and if they did, they were not typically enthusiastic about using it. (Most users avoid Advanced Search on websites. Just calling something “Advanced” scares most of us away.)

Training lingo

Another problem was evident from observing and listening to the testers – the use of training lingo in field labels and menus, such as “ILT,” “Training Activity Type,” “Roster,” and so on. Most of the testers didn’t understand these terms and would sometimes make an incorrect choice because of it.

Course registration

Testers stumbled through the process of course registration and were often unsure if they had actually registered. After the trouble they experienced finding a course, this compounded their frustration. The key concerns the testers expressed were uncertainty about where they were in the registration process, how many steps they needed to complete, and whether they had successfully registered. Just as with the shopping cart and checkout experiences on retail websites, the course registration process in LMS’s requires clear signposting and lots of feedback to let learners know where they are, what they need to do next, and whether they have been successful.

What you can do

Based on this high-level summary, it’s obvious there was a lot of room for improvement in the usability of this client’s LMS – in the labeling and presentation of search and navigation options, in the search logic, in the registration process, and in terminology.

And as we recommended to our client’s LMS vendor, we believe one of the most useful features an LMS can provide is faceted search (also known as guided navigation). For users looking for learning materials, a well-implemented faceted search feature would be a huge help. (The vendor said they were working on it.)

But this leads to a key question: “Even if I do test the usability of our LMS, what can I do about the issues the test uncovers? It’s a vendor product. We don’t want to do significant customization because that will complicate future upgrades.”

If you are working with a vendor system, as opposed to an open-source product, it’s true that not all of the power is in your hands. However, here are few options to consider:

• If a problem is within your scope, as a client, to configure, then you can address it in house. This is usually the case with labels and terms that learners have trouble understanding. And terminology is often a huge part of usability.
• Invite vendor reps to observe usability testing – there is no better catalyst for change. This is preferable to simply reporting the issues to the vendor, because when vendors see real users experience real issues, the impact is much greater. It’s hard for them to try to deny or explain away this evidence. (If the vendor can’t be there in person, send them video clips.) In addition, make sure your vendor has a user forum and make your voice heard.
• With issues that you know the vendor is unlikely to address any time soon because of their complexity (such as issues pertaining to search functionality), you can at least create quick reference cards or other guidelines for instructors and learners. If nothing else, by uncovering the specific tasks or process steps that learners struggle with, you can create more targeted and helpful user assistance material.
• Finally, don’t let LMS vendors off the hook – let them know what is frustrating end users, and slowing their productivity. Tell them that the learner experience has to become more of a priority.