A/B Testing vs. UX Research

What is A/B Testing? What is UX research? How they are interconnected?

Whether you are developing a new piece of software, a company website, or a physical product, testing and analysis play a pivotal role. The development process of an individual product or a service can take months if not years to be completed properly.

While there are variations and deviations of the core concepts, developers mostly lean on two types of testing and research. With that said, let’s take a look at what A/B testing and UX research are all about, how they complement each other and how they can be utilized to their fullest potential.

The significance of product development research

Product developers, UX designers, software programmers, and others have always faced the issue of having to patch and fix their products after launch. The amount of fixing and maintenance work needed after developing a piece of code depends solely on the quality of the workflow and development pipeline.

As such, implementing a specific testing method is just as important as choosing what programming language or visual style to use. Some of the most important reasons for testing and research before launch include:

• Detecting bugs, glitches, and otherwise unplanned behavior in your UX design.
• The cost of fixing bugs before launch is much smaller than afterward.
• Using a specific testing or research method gives your development team an opportunity to learn something new that they otherwise wouldn’t notice.
• The performance and quality of your design can be significantly raised if you involve test groups.
• Spreading word of mouth about your website or software application is always positive before launch.

Testing and Research 101 (Introduction to A/B testing and UX research)

Moving from the alpha stage of your development to beta and public testing is exciting. All those weeks of developing a product will pay off and you will finally be able to show people what you have been working on. However, as soon as you and your team get your hands on a concrete deliverable, things start to get complicated.

Choosing the proper method of analysis and feedback is never easy and it all depends on the type of product you have created. This is where the two most popular methods of gathering data come into play, and most teams choose one or the other as their tool of choice

What exactly is A/B testing?

“A/B testing is the act of running a simultaneous experiment between two or more pages to see which performs or converts the best. Despite the name (A/B testing), the experiment can be conducted with as many pages as desired.” — Unbounce

A/B testing might sound complicated at first, but it’s far from it. This type of testing represents creating two variants of your product and letting the test group experience them one after another. Whatever results they gather from both of these variations will make one more comfortable than the other in their eyes.

There are a plethora of possible results in A/B testing including complimentary qualities (where both products are good) and double negatives (where both are bad). This type of testing is often implemented by UX teams that optimize software products and apps. It is also used extensively on products that serve an internal company purpose.

For example, Electronic Arts used A/B testing when they launched one of their flagship titles — SimCity 5. They focused on digital sales of their software (a game in their case) by introducing two different types of pre-order offer displays on their site.

The resulting sales were higher as a result of choosing the right variation of their pre-order website design. The main issue with A/B testing is that most developers create variations based on opinions and internal agreements — not on the previously conducted research.

What is UX research?

“ UX (user experience) research is the systematic investigation of users and their requirements, in order to add context and insight into the process of designing the user experience. UX research employs a variety of techniques, tools, and methodologies to reach conclusions, determine facts, and uncover problems, thereby revealing valuable information which can be fed into the design process.” — Interaction Design Foundation

UX research takes a different approach as compared to A/B testing. Here the development team creates a single variant of their product and presents it to the public. Then user behavior and inputs are recorded and analyzed, to get insights about the product. This type of research goes into far more detail and analysis than A/B testing, with concrete feedback and thoughts collected from test subjects.

Most UX research takes place before the product is fully developed. This allows developers to listen to test groups but also keep working on the product regardless. This type of feedback is often implemented in digital online products such as website development— clients review finished websites and give reviews to the developers/designers.

No user experience is perfect — every individual will have their own feedback to submit. This effectively means that there is always room for improvement and optimization on the developer’s part.

The major flaw of UX research is that data can often become too convoluted and stray away from the initial goal. The analysis done by the developer needs to be as systematically implemented as possible in order to get actionable results.

Two Sides of a Coin

When we look at the definitions of A/B testing and UX research we can easily draw parallels between the two. Both of these review methodologies were created with a singular goal in mind — creating better products. These methodologies can be used in tandem more often than not.

One such way is the incorporation of some specialized applications for review purposes. If you are looking for a tool that can help you use both these methods, zipBoard has you covered. This platform was designed with the goal of offering options and capturing user inputs, that complement A/B testing requirements as well as UX research.

Your initial builds can be analyzed with zipBoard and you can use the collected information to finish your product on time. Focusing on the design and production aspects of your product creation cycle is far easier with a proper tool in place.

There is no need to separate and differentiate UX research and A/B Testing, but rather use the best of both worlds to ensure a good quality of service for your clients. Let’s take a look at a workflow — how we can use both A/B testing and UX research simultaneously, in order to get relevant insights for our development team to work on.

Combining Both Methods

1. Using UX research to identify potential issues in your design, interface or visual appeal can be a good way to narrow the A/B testing choices for your test group. Present your group with a UX research platform and analyze the data carefully. Make sure to limit your final choice of variations to two-to-three and never go above that. The more choices you give to your audience, the more confused they will be.

2. A/B testing can act as a second stage to your development cycle. Since you have eliminated most of the unobtrusive issues in the UX research data analysis phase, you can safely move on to A/B testing in a much clearer direction. This will allow your test group to be presented with far more refined, well-developed, and bug-free builds of your product.

3. Going back to UX research for the final phase of your design cycle is always a good idea. Once you have built your beta build and are ready for further testing, make sure to conduct one final UX research cycle before moving to production. This time, focus on the build that passed A/B testing with the highest approval ratings.

The other builds can serve as secondary backups if the representative build fails to meet expectations. A/B testing is done at this point and you don’t have to go back to it — it would only confuse your development team.

Implement and Innovate

Depending on the type of development you are involved in, you will always find a different way to implement these two review systems. There is no need to solely use one or the other — this will only cause you to end up with half-baked results and poorly optimized products.

Take your time to discover ways in which general research can precede concrete comparative testing. Only then you will be able to refine new research methods and innovate in your niche.

This a guest post by Leona Henrynson from Essay Supply.