Why Card Sorting Doesn't Work: Create and Test Taxonomies Instead

Let me start out by saying that "doesn't work" may be too strong of a phrase here. There are valuable things that can be learned from card sorting. However, I would suggest that you will be far better off having a information architecture expert create a taxonomy or site map for you and then use that to test users. This " Create and Test" method is especially useful for large, deep taxonomies.

The reasons to have a knowledgeable expert create a taxonomy first, and then test it are the following:
1. You are not asking people who know little to nothing about usability or logically organizing information to create your site structure for you.
2. What is card sorting really? It is having a number of people give suggestions of possible groupings and possible labeling of those groupings. I would argue that you can get the same kind of feedback while users test your taxonomy.
3. Usability testing time required to test a taxonomy is far shorter than the time to do a card sorting exercise. During testing you will be watching users, measuring their success rates at finding the information they are looking for, and hearing and seeing feedback as to what issues they are having with naming and/or organization. What do you have at the end of card sorting? An oftentimes, very subjective set of data, the statistical significance of which is questionable if you are not testing hundreds of users. It is possible to test hundreds of users at once using online tests. I would argue that the online tests are far more valid that small sample sizes of 10 and under.

Some suggestions on the Create and Test method are:
1. Try to limit each category within your taxonomy to having 7-10 items if possible.
2. Create a click-through prototype of the taxonomy. This is not a full-blown site, but simply the categories listed as blue links on a page and when you click one, it displays the next level of categories. This allows you to see users interacting and allows them to talk through what they are thinking. It also helps you see if they were successful on the first attempt, second attempt or not at all.
3. Test a range of items or questions. Since you are doing a less time-consuming click-through test of the taxonomy itself, you can have the user find 10-15 things that would be contained within the taxonomy. If it is a product taxonomy, for instance, test some of your best known products, some obscure products, and other products where you are a unsure as to whether or not the taxonomy is solid. If you are testing a support site, maybe you give the user a list of 10-15 FAQs that most people seeking support ask and have them click through to where they think the answers are. Be creative here.

In the end, if you use the Create and Test approach, you will:
1. Get valuable feedback on naming and trouble points in your taxonomy.
2. Have strong measurable data about user success rates.
3. Have completed the test (and analysis) in a far shorter timeframe than with card sorting.

Overall, card sorting is effective in finding synonymous names for categories and identifying user agreement on category names that are strongly indicative of the users' mental models. However, results are often not directly applicable because users are not as familiar with all of the content on the site, users don't think of the confusing synonyms that could be a result of the words they choose, their categories sometimes mix unlike things in a confusing fashion, among other issues. In contrast, the Create and Test approach gets you much of the same feedback, without the longer timeframe, and gives you measurable success results rather than possibly applicable yet conflicting and not well though out suggestions.

If you liked this article: Maybe you'd like to hire us at:

The Usability Review

Subscribe to This Blog (RSS)

The Myth of Limiting the Number of Clicks

Time after time, you hear clients and even designers say we need to get the user to their destination in 2 or 3 clicks. The end result of this point of view is usually a home page with so much stuff on it that you can't find anything you need OR a navigation scheme that has 50 items top level category. Most of the time, neither of these approaches is efficient or effective.

So, what is the right approach? Well, my first instinct is to refer to an article written on Jared Spool's site www.uie.com. Jared Spool is one of the most well-known usability and user interface experts around. The name of the article is The Truth About Download Time.

What does download time have to do with number of clicks you ask? The article is all about user perception versus reality. The basic premise of the article is that user's perception of the speed of a site were strongly correlated with their successful task completion on the site and not at all correlated with the actual speed it took to load pages on the site. In fact, the "fastest" sites according to the test participants actually had the slowest download speeds. Now the article was written in 2001 when download speed was very important as bandwidth constraints freqently made us wait for pages to load.

I would extrapolate these results to a user navigating through a deep site taxonomy or through a series of tasks. When designers and clients force too much stuff up to the home page or put too many links in a category, they are actually making it MORE difficult for users to complete their tasks. When site maps and taxonomies are designed such that there are fewer options that are more clearly identifiable, task completion will be improved. I usually try to keep categories to 7-10 items.  A good reference point is the classic journal article by Miller in 1956: The Magical Number Seven, Plus or Minus Two.

There are those who would argue that the use of Miller's paper with respect to navigation is a misinterpretation and that far more links (and presumably fewer clicks) will work just fine.  But what has often been absent from discussions about limiting the number of categories in a taxonomy is users employing the process of elimination during navigation.  Process of elimination of the incorrect options is far easier with a small set of options.  In fact, I recently observed a usability test session where the main menu had only 5 options.  This menu had a lot of usability problems to begin with: 1) the options were hidden in a rollover by default; 2) the client's design firm had used icons instead of labels (the latter were only visible on rollover); 3) the nomenclature was not very clear for each option; and 4) the placement of the rollover menu was in the upper right of the screen (one of the last places users typically look).  Yet, despite all of these negative usability factors, the menu was largely successful.  I attribute that to the fact that there were very few options and users were able to single out choices by process of elimination (even though the couldn't read the labels on the icons and had to keep the choices in memory).  The argument to use much larger numbers of links also largely ignores the time time it takes to read those options and the memory required to recall the possible options that could be the right one.   Also, users often click the first option they find that sounds right.  In many cases, the greater the number of links displayed, the greater the likelihood that there is some overlap within those links which can lead to users exploring the wrong path first and wasting even more time.

There are several caveats to the above situation, though:
1. The number of clicks should be reasonable. If you are forcing people through 15 levels of a taxonomy, they better be doing their taxes or something complex that they have a strong motivation to complete.  I am not a proponent of making navigation paths any longer than necessary.  For instance, on the Lenovo.com computer site, their new customization screen for purchasing a laptop has a carousel with 39 options.  You literally have to click the next button 39 times see each option.  And this is only for the "System Compoents" tab.  There are more options for the other 3 tabs.  And while you can skip around, who knows what you are missing.  They should take heed of the Dell example which has more options per page because who knows what you are missing in terms of configuration if you aren't clicking through the 39 options.  It could be something important to you.

2. If the list of links are things that most people are familiar with, you can get away with more links. US states for a US-centric site, or city names within a state are examples. For groups of things that are more subjective (like product categories), it is best to stick to ~7-10 links.
3. You must do a good job of creating descriptive, easy-to-undestand, mutually exclusive categories. "Descriptive" does not mean make the category name 8 words long. It should be as short as possible, yet meaningful. "Mutually exclusive" means that a user should not be confused between two categories when looking for an item within the taxonomy. There are cases where items will appear in two categories (or more) within a taxonomy, but those cases should be minimal.

While The Truth About Download Time is not specifically about building taxonomies and site architectures, it IS all about user perception and the correlation of that perception with site performance. I think it is a great analogy to the number of clicks issue. So, the next time you hear someone say "Everything on the site should only be two clicks from the home page", you will have some sound reasoning about why that is not always the case.

If you liked this article: Maybe you'd like to hire us at:

The Usability Review

Subscribe to This Blog (RSS)

To Pop or Not to Pop: When to Use Pop Up Windows

A question that frequently comes up during web site design and implementation projects is if and when to use pop up windows. When I refer to pop up windows, I am not referring to the light boxes or other dialogs that display to ask a question that you need to answer to continue or cancel a process.  Instead I am referring to popping up little (or large) windows that contain additional necessary information to continue or supplement what is going on within the context of what the user is currently doing.

So, the simple answer of when to use pop ups is: virtually never. But if you choose to use them, here are some of the basic pitfalls to avoid:

1. Not including browser controls in a pop up window.
2. Putting links in a small pop up window.
3. Putting a large amount of content in a tiny pop up window.
4. Breaking the user flow (and back button) by launching new windows with every click of the mouse.
5. Search engines will not index the content.

These are just a few of the flaws that sites oftentimes contain with respect to popping up windows. Lets explore a few of them before discussing the very limited set of circumstances where you should use pop-ups.

The first is not including browser controls on a pop up window. This is a substantial usability error. First, if you are going to provide any links within the window, how is a user supposed to get back to the previous page. The back button is the most used button on any piece of software in the world. Why break it? Sure the user can use the right click menu, but most users either don't know about it or wouldn't think of it and Mac most Mac users don't have that option. Futhermore, what if the page failed to load correctly for some reason? We have all had this happen. The user would not be able to refresh the page. The Sprint PCS site is a great example of this. The picture shows a logged in view of my account. When a user clicks to view his bills he gets a small pop up that looks like this picture:



When one of the links is clicked a PDF (an issue for another day) is displayed in the tiny little browser window. If the wrong bill is chosen or the user was not sure of the month of the transaction of interest, he would have to then close the window, re-pop up the window, re-launch a different PDF and hope that was the right one. Not a good user experience.

The second and third issues are that people put links to other content in small pop up windows. The Sprint example works here as well, although you see this error on virtually every site with pop ups. Am I really going to view my entire bill in a tiny little window? No. The user can maximize the window or increase the size of the window if they want you say? Why bother making them? Why not just use the browser in the first place and not launch a new window? The reality is that it is probably due to a default setting in the backend billing system they use (maybe Checkfree), but that is no excuse not to fix it.

Another typical usability error is to automatically pop up a window. For years, CNN.com launched a pop up to allow users to choose US or International edition and pop up blockers prevented it from being seen. They have finally fixed this in their recent redesign and simply show it on the page:



Another major usability flaw with respect to pop ups is to continually launch new windows breaking the user's ability to easily use the back button. Launching one new window is one thing. But if you are launching more windows than that you are forcing the user to figure out which window which content is in. Odds are they have other windows open on their computer as well. Furthermore, you break the back button and force users to constantly move the mouse to the upper right corner to close the window and then back down in the the browser to interact with the page.  I once watched an incredibly web-savvy ad agency executive be completely confused by constantly clicking a link that was opening in another tab on the browser.  He did not realize this and eventually closed the whole browser down and re-opened everything...during the middle of a presentation.  If a web-savvy exec is having trouble, how about your less skilled users?

And, finally, the last issue with pop ups is that the content within pop ups will not be indexed by search engines unless it is linked to elsewhere on the site.

There are, of course, a LIMITED set of circumstances under which popping up a window is acceptable. I hesitate to say pop up as well in that simply using the link attribute 'target="_blank"' to "launch" a new window is often a better approach because javascript pop ups are often blocked. The circumstances are as follows:

First, the amount of information in the pop up is so small that it fits in a small pop up and has no links. Good examples are the definition of a word or simple contact information or a small tip about using a site. However, if possible, I would suggest using a rollover for this type of information so that a user doesn't even have to bother clicking a link.

Second, if there is a need to preview content or compare something side-by-side and it won't fit in the same browser pane, then it is acceptable to pop up a new window. For instance, when editing your resume on Monster.com or configuring something that you need to see how it looks (like "Previewing" your blog which blogger.com actually does in the same pane these days).

Third, if the user has done a significant amount of work to configure a page (say running a detailed flight search on Kayak.com), and there is a chance that work may be lost or the page is better served as a jumping off point to investigate multiple options, then using a pop up may be appropriate.

Overall, the web is littered with pop ups that yield poor user experience. When in doubt, don't use them. If its good enough for Google not to use then, then it should be good enough, in most cases, for your site as well.

If you liked this article: Maybe you'd like to hire us at:

The Usability Review

Subscribe to This Blog (RSS)

Right Navigation vs. Left Navigation

There has been a very long-standing debate as to whether navigation is as effective on the left side of a browser pane versus the right side. There have been various studies that have found conflicting conclusions. One study that was originally done by Razorfish in Germany found that right navigation can be more efficient than left navigation (See study here). One of the core arguments in that study is that because the site navigation is located near the browser vertical scroll bar, there is less distance to move the mouse from a right navigation bar to move said scroll bar. Poynter Eye Tracking Studies also found that there were more fixations on right navigation, however, they were unsure whether that was due to the novelty factor since most web sites use left navigation.

My opinion is that placing the secondary or primary nav in the right navigation area is about the last place I would put it other than below the fold on the bottom of the page and out of sight. There are a number of reasons not to use this placement:

1. Many user visits start at search engines where, frequently, ads fill the right column.
2. Users are accustomed to finding related information on the right hand side of pages.
3. The right hand side of the page is typically the last area users look toward during a visit.
4. The right navigation can get cut off by the right edge of browsers (especially with tablet and mobile devices).
5. The advent of the scrollable mouse makes gains in efficiency of the right hand browser scrollbar moot.
6. Back Button placement.

Let's address these issues one by one.

One of the strongest arguments for not having right hand navigation is that many users start their visits from search engines.  On many search engines the right hand column is filled with ads.  In other words, many users have conditioned themselves not to look towards the right column.  Likewise, in point 2 above, many sites use the right column for related material from other sections of the site.  In a similar way to point 1, this conditions users not to look to the right column for things that are immediately relevant to what they are trying to find.

The third point is summed up by the following heat map summary image from the Poynter study:



Users go a lot of places before looking at the right side much.  In fact, in a recent usability study of some web conferencing application, I observed that one participant did not locate the right nav until 30 minutes into a 60 minute session, even though there were 2 other participants in the test with him talking to him the entire time.

Perhaps the most important point, however, is the perfect conditions under which many of these studies are performed. Browser widths are set prior to usability tests to be optimal width. This automatically eliminates the possibility that the menu will be cut off the right side of the screen.  In reality, there are inexperienced users who leave their windows set smaller than most or don't even realize they are set too narrowly. In these cases, we basically ensure that the user won't see right hand navigation because it will be cut off the right-hand side. That will never happen with left-hand navigation. Even more importantly, mobile web and tablet usage is increasing dramatically.  The devices have smaller screens, many of which ensure the navigation being cut off the right hand side.  Certainly this issue can be overcome with proper coding, but how many web sites do you know that are coded perfectly to size correctly with browser width? And if it's crunch time to deliver a site, do you think developers and product managers (who are likely to have large monitors) are paying attention to these types of issues?

To cap it off, the browser back button is probably the most used button on any piece of software in the world. It is to the opposite side of the right nav meaning greater distance to use it in a right nav situation, and decreased usability. It is important, as is the fact that having a scrollable mouse makes the efficiency gain in right nav placement being close to the scrollbar moot in many cases. Just think how annoying something as simple as Internet Explorer's placement of the refresh button on the opposite side of the browser from the back button. Notice how Chome and Firefox don't take that approach.

Overall, in designing I think we have to go with the lowest common denominator: User Success! Potentially having the right hand navigation cut off of the right hand side of the screen is not an acceptable option to ensure user success; nor is not locating the navigation in the few seconds of visiting the site because it is the last place one looks on a page. Thus, left navigation, in my view, is the far more viable navigation option than a right side navigation.

If you liked this article: Maybe you'd like to hire us at:

The Usability Review

Subscribe to This Blog (RSS)