Usability Testing – Part 2

In my last post, I shared my experience with UserTesting.com, a website that offers cheap, crowd-sourced usability testing.  In this post, I’ll be sharing with you the results I received for my site as well as my comments on the actual video (I’ve watched it several times!)

The user started out telling me a little bit about herself, which I really enjoyed hearing.  I immediately identified her as a great potential user for my site, as she loves to cook and cooks for her husband and three kids.  She already uses online recipe websites, and seems to be adventurous in the kitchen; always on the lookout for something new to try.  These are all fantastic qualities in a potential user, so either I got lucky or UserTesting.com does a great job matching users to each project.

Right away, I was disappointed by the way my website looks on her computer.  She was using 1024×768 resolution, and also had various IE toolbars enabled so my site looked a bit squashed.  It might just be that I’m not used to seeing it this way, since I run a super-duper high resolution on all my monitors that only a bald eagle wearing a loupe could possibly make any sense of.  She didn’t seem to mind too much though.

She also loved the meatball photo on the home page (I guess I have Merix to thank for that one!) and gave it quite a lot of praise.  She had absolutely no problems at all figuring out how to register for the site and logon.  She chose not to use the Facebook “one-click” logon; either she didn’t notice it or she’s not a Facebook user.  It’s a curious stat, as I’ve noticed a very low percentage of my users have linked to their Facebook account, even though it provides such a simple way to logon and doesn’t require users to remember their password.  It makes me think twice about adding other logon mechanisms, such as OpenID.

She also wasn’t too clear on what the various top menu options were, especially “New Recipe.”  I believe I can fix this by providing tooltips on each option.  The drop down triangles also confused her, however I’ve heard that feedback several times now and have a bug logged to address that issue.  I do recall a discussion on whether we should say “New Recipe” or “Upload Recipe”, but we eventually settled on New since Upload had too much of a technical connotation, like you might have to upload files or something.

The next step was to find recipes.  I was somewhat surprised her intuition was to click the “Find Recipes” circle icon, but I guess she was drawn to that area through the power of the delicious meatballs.  I figured for sure she’d type in something into the “Quick Search” on the upper right, which for all I know was completely invisible to her.

It’s perhaps a good thing she used the advanced search feature, since she provided some feedback on the full search page.  It was very clear to me she wasn’t too impressed by the search capabilities of the site.  I believe she was hoping for some immediate results, such as top rated recipes on the site or the ability to easily browse by tag.  She made it clear that she has a desire to be inspired by a cooking website, and wants creative suggestions rather than being asked to fill out search forms.  I would love to accomodate this sort of feedback on the site, but for the beta release I had to stick with a very simple search page.

On the plus side, she had no problems using the search page; she typed in chicken into the keywords and ignored the other search parameters.  The results page was something she also wasn’t too pleased with.  In fact, I’ll go out on a limb and say she’d have left the site at that point in a real world scenario.  First, she was upset she couldn’t sort the results.  This is a feature I’ve been meaning to add as well, but just not something I’ve had time to do yet.  She mentioned she’d rather see the highest rated recipes at the top of the list; however, little does she know this is the default sorting, it just happens to be that no one has rated any of my chicken recipes yet!  She also had the impression that my site doesn’t have that many recipes (she mentioned this twice), however I believe this is because I will only display a maximum of 100 results for any search so that I don’t completely flood the user.  She seems to think I only have 100 chicken recipes, when in reality I probably have many times that amount.  I have a few ideas in mine on how to fix this.  For example, I can say “Displaying top 100 out of xxx recipes” at the top.  Even better would be the ability to “page” through the result set, no matter how many recipes turn up.  With this said, I still can’t really brag about the number of recipes I have.  We just hit 10,000 recipes recently (yay!), where-as the other big sites have hundreds of thousands or even millions.  To be fair, it took them years to build this amount of content.

She also had some problems managing such a large result set (which makes me glad I capped this at 100!)  She became quite frustrated looking for recipes that she was interested in.  I think her mind works best by scrolling through the whole list and remembering a few that she wants to go back to look at, but then when she scrolls back to try to find them she loses her place.  Had the results been organized in some logical manner, I think she would have found this easier.  Another idea I had is to put “checkboxes” next to each result that the user could check if they wanted to remember certain recipes in the list.  However, random checkboxes that had no other purpose might tend to be even more confusing.  I cringed during one spot where she closed the recipe viewer and it jumped to the top of the page, making her once again lose her place.  In my defense, that’s actually a bug and not supposed to happen.  I actually went through quite a bit of effort to make sure I wouldn’t lose your place on the page, since I hate it when sites do that to me.  This was actually the entire point of doing a popup recipe viewer rather than navigating to a new page.  I will need to fix this at once!

The next task was to add two recipes to the calendar.  I was absolutely shocked when her first reaction was to drag the recipe to the calendar bar!  Dragging and dropping is a meme that has simply never caught on in the web world, and even a feature I was thinking about cutting since I figured no one uses it.  However, it was pretty cool that my site accommodated what came naturally to her.  Using this method, she had no problem adding a recipe to the calendar.

She then decided to search for fish recipes.  Rather than hitting back, she found the much smaller Change criteria button which I found interesting.  I’ve always wondered why so many web pages have Back links on them when you can just as easily use the Back button on your browser, I guess now I know.

When adding the second recipe to the calendar, she decided to see if there was a button she could use instead of dragging.  She had some problems finding this, but at the last second she noticed the action toolbar under the recipe image.  I guess these buttons are somewhat easy to miss.

The next step was to go to the calendar, which she had no problems finding.  She immediately complained about the calendar layout, and frankly I agree with her 100%.  I had quite a long debate about this page with my designer, but eventually lost.  I’ve simply never liked the look of this page.  The problem is fitting a seven column calendar on the page while also having the shopping list widget displayed is simply not doable without horizontal scrollbars (yuck!)  If I could take up the full screen (eg Google Calendar,) it might be viable but I just haven’t found a good layout that works.  To be honest, I’m having second thoughts about the calendar design as a whole.  I have a theory that very few people even want to plan meals to the day.  It might be a better idea for people to just drag recipes into buckets for “This week” or “next week” and not have specific days.  Perhaps the calendar could even be customizable where users can configure their calendar to display individual days, individual months, or just a freeform “drop box” of random recipes they plan to cook in the near future.  Certainly some more thought needs to go into this page, since I think very few users need disciplined meal scheduling that is accurate to the day.

My tester also had no problems creating a shopping list.  She seemed to ignore the checkboxes that allow you to select individual days or individual recipes, but quickly found the “Create shopping list for this week” link at the very button and found this to be the most intuitive way to create a shopping list.  When she clicked it, I think she was confused that she received no UI feedback.  Since she was scrolled to the bottom of the page, she didn’t see that the shopping list widget was populated.  I think I should popup an alert saying “Shopping list created!” or something, and perhaps automatically scroll back to the top of the page so users will see the items in the list.

The final action was to print the shopping list, which she had absolutely no problems doing.

Though she had some good comments on the site, I don’t think I’ll be seeing her again.  She seemed to view my site as a recipe database, and compared it to AllRecipes, rather than seeing it as a meal planning tool (which is perhaps not a tool she really needs.)  She said she was not certain if the shopping list feature would be useful to her, as it might contain a lot of ingredients she already has.  She might just be someone who prefers to jot down a shopping list on paper before she goes to the store.  She did say she plans her meals in advance though, which leads me to believe she might use the site if it did more to inspire her culinary creativity by intelligently suggesting recipes and allowing her to browse a lot easier.

I think I need to decide if KitchenPC is really a meal planning tool, allowing busy soccer-moms to plan their recipes and organize their shopping list, or a recipe portal to inspire chefs and expand their culinary horizons.  I can see a lot of directions I could go if I wanted to attract the latter type of user.  My meatball picture could be user-submitted photos of the week, and have links to the featured recipe.  I could have video interviews with chefs, cooking articles, and do a lot more on the home page to call out featured recipes and content.  However, so many sites are already out there that do such an awesome job at this, I’m not sure if I’m really in the position to compete in that market.  I have, however, noticed that pretty much no one out there (including AllRecipes) can really do meal scheduling worth a damn so I’m hoping to really focus on this and do it right.  Though I’m a ways off, I’m still quite convinced there’s a market of busy parents who want to plan a week’s worth of meals at a time, go grocery shopping once on Sunday, and stick to a plan during the week.  If I can make this scenario brain-dead simple to use and attract those sorts of users, then I think I have myself a viable business model.

Lastly, I’ve decided to post the entire video on my blog for your enjoyment.  I checked UserTesting.com’s EULA to see if they had any issues sharing this content, and it turns out they not only allow this, they seem to encourage it!  In fact, they provide HTML you can embed on your site to share the video.  I decided to just host it myself since their servers are kinda slow, plus I also wanted to be polite and blur out my tester’s email address since she might not appreciate me sharing her PII on my blog :)

Enjoy!