The new homepage design was all ready to go, and we were excited to launch. As the team discussed our rollout plan, we all agreed that testing the design before launch would be the right thing to do—we are, after all, focused on user-centered design. The problem was doing testing in a timely manner, while still gathering enough information to make sure our new homepage would provide a fantastic experience for our readers. 

Believe it or not, UX Booth has a pretty small staff (see for yourself on our About page). And in addition to our work with all of you, we all work full time in the UX field. So adding usability test creation, recruitment, moderation, and analysis to our busy schedules was an overwhelming prospect. Plus, we were feeling the pressure to launch our new homepage design quickly, so that we could share it with you.

Of course, there’s more than one way to approach testing. We looked through some articles for recommendations:

• We could do remote testing, making recruitment easier
• Remote moderated testing offered some clear benefits
• Guerilla usability testing would give us quick results

But ultimately, moderation was still going to be a struggle. So we turned to our trusty Complete Beginner’s Guide to Research for inspiration.

Enter UserZoom

We’ve known the team at UserZoom for quite awhile. We’ve bounced around a few article topics – sometime soon you may see a UX Booth article from one of their team members – and we featured them in our Complete Beginner’s Guide in 2016. Here’s what we had to say about them:

The good news is, whatever you need, UserZoom has it. Usability testing, both moderated and unmoderated, remote testing for mobile and desktop, benchmarking, card sorting, tree testing, surveys, and rankings: they’ve got it! The bad news, as can be expected with any product this robust, is that it can be overwhelming to learn, and it is expensive. Still, for organizations with the budget to handle it, UserZoom is a solid, effective choice.

Luckily for us, the team at UserZoom is fantastic. Alfonso and Phil got us set up, and Ann was there along the way to answer our questions and provide advice. In just over two weeks, we were able to set up our test, recruit participants, get results, and analyze them. 

The Process

Here’s a step-by-step of how it went down.

Day 1: Before even looking at UserZoom, our team at UX Booth discussed the study goals, and we identified five introductory questions, six questions about our current homepage, and six questions about our new design.

Day 3: Having finalized our questions, we reached out to UserZoom for help setting up the test. It turned out to be very easy. We chose to run an “Agile Usability Test (Think-Out-Loud)”, and the site led us through a set of simple steps to prepare the test. 

1. Choose the type of study (Agile Usability Test, in our case)
2. Select the device you expect testers to use to access the study (desktop or mobile)
3. Name the project and describe the test’s goals – a smart thing to ask, as it ensures we know those goals
4. Choose how to find participants – in our case, we were recruiting our own, but UserZoom also offers the option of finding participants for you
5. Identify user profiles (useful if you have multiple personas or audience segments to screen for)
6. Add tasks and questions
7. Review and preview the test (from the participant perspective)
8. Launch!

The whole process took less than 30 minutes.

Day 4: We held off on launching the test until after recruitment, which took the form of a few tweets and an article to let our audience know we were redesigning. Which reminds me – thank you to the many wonderful readers who responded and participated!

Day 8: We launched the test four days later and shared it with the volunteers who had responded to our article. We gave them one week to complete the test. Shout out to UserZoom – when Ann saw we had launched our test, she reached out to ask how things were going, and she pointed out a few additional features I hadn’t known about.

Day 15: One week later, we had our results. We hadn’t needed to moderate the test because UserZoom does such a great job of videoing the interaction, recording audio, and pulling the written responses to questions into an easy-to-scan Excel doc.

The best part? It only took about two hours to review the responses and reach conclusions, and the conclusions were varied and valuable. We found that some of our assumptions were dead on – yes, you love having article categorized by topic, and our new design could make it easier for you to find the most recent articles related to trends or sub-sets of UX. But some areas we were proposing changes weren’t as helpful as we’d expected – you helped us recognize flaws in our navigation, and pointed out that the new homepage could be awfully confusing if we didn’t provide enough text.

Now we’re onto phase 2: updating the designs with your feedback, and then onto launch!

Easy and Useful

Ideally, testing a new design should be easy to do, and it should provide valuable feedback. With UserZoom, we were able to do both – even on a tight timeline, with limited availability. We recommended UserZoom before based on individual experiences and stories from our readers. Now we recommend them even more, based on our experience at UX Booth.