How we use A/B testing to find out which ads work best. In the process of choosing the best site for conducting experiments, we tested and compiled a selection of services for testing. With helpful comments. And we share with you!

What functionality was important to us?

• The ability to test the service for free in full force. That is, on a real advertising campaign with large flows of visitors.
• Possibility to upload your version of the page. This means that we want to create our own beautiful page and upload it to the service, make changes in the online editor there and run the test.
• Convenient and non-buggy online page editor.
• Correct hooking of testing goals. Simple goals can be configured directly in the services. Actually, according to them and to intermediate goals, you can compare the conversions of options and choose the winning one.

How to sort out the collection?

• Service- the address of the service, everything is simple here.
• A/B or MVT- which of the testing options supports the service.

1. A/B testing is when the variants of the tested page differ by one variable (different button, different title, etc.). . For example, in one version the “Order” button is red, in the other it is green. In general, the pages are identical.
2. MVT is multivariate testing. When variations of the page being tested differ in many different ways. The first version is blue, with a red "Order" button and a pattern of butterflies in the page header. The second option is purple, with a green "Order" button and a pattern of unicorns. The third option is generally black and white, with a blinking “Order” button and a photograph of the customer in the background.

Some landing page services only support A/B testing, some both A/B and MVT.

• Tariff plans- how much money the service wants for its services. What parameters determine the coolness of the tariff plan?

1. The number of visitors that can go to the page (if there are more visitors than the tariff plan allows, the experiment is disabled).
2. The number of domains hosting landing pages.
3. The number of experiments themselves that can be run in the service.
4. The number of accounts that can be created on the service.

Different tariffs include different parameters, here what kind of service is worth much.

• Templates\online editor- Somewhere there are ready-made templates, somewhere there is an editor with which you can change variables.
• Demo version- how many days you can use this service for free.
• Comments- our personal comments about the work of each service.

A selection of services for A/B testing

Russian-language services

Service #1

• Service: http://abtest.ru/
• Testing support: A/B
• Tariff plans: testing is free, the service is dying and is in beta.
• Templates\editor: online editor.
• Comments: Allows you to upload your landing pages. Terribly buggy editor.

Service #2

• Service: http://lpgenerator.ru/
• Testing support: A/B.
• Tariff plans:

1. $37 , 3500 visitors, 2 domains, 25 pages.
2. $58 , 9000 visitors, 5 domains, 50 pages.
3. $119 , traffic, number of domains and number of pages is not limited, connecting your own domains.
4. $440 , traffic, the number of domains and pages is not limited, connecting your own domains, 15 client accounts, personal branding.

• Templates\Editor: templates and online editor.
• Demo version: 14 days.
• Comments: There is a set of templates and paid and free. There was no tool to upload your own sites to edit in the editor. The editor is simple but takes time to learn.

English language services

Service #3

• Service: http://unbounce.com/
• Testing support: A/B.
• Tariff plans:

1. $49 — 5,000 visitors.
2. $99 - 25,000 visitors.
3. $199 — 200,000 visitors

On all plans: unlimited number of tests and pages in all plans, statistics, online page builder.

• Templates\Editor: templates and online constructor(!).
• Demo version: 30 days.
• Comments: The editor is quite convenient and pleasant to use. Didn't find how to upload your page for editing. Page templates are dull and uninteresting. The service has not just an online block editor, but a full-fledged online constructor with which you can create a simple website from scratch.

Service #4

• Service: https://vwo.com/
• Support testedand I: A/B, MVT and mobile app testing.
• Tariff plans:

1. $49 , 10,000 visitors.
2. $129 , 30,000 visitors.
3. Individual tariff plan- about a million visitors a month, testing on mobile, linked accounts, personal consultant.

• Templates\editor: online editor.
• Demo version: 14 days.
• Comments: the service has a beautiful interface, but there are problems when loading its page.

Service #5

• Service: http://www.convert.com/.
• Testing support: A/B&MVT
• Tariff plans:

1. $9 , 2000 visitors.
2. $29 , 10,000 visitors, MVT.
3. $59 , 30,000 visitors, MVT.
4. $99 , 50,000 visitors, MVT + mobile site testing.
5. $139-$1499 , tariff plans for agencies. Unlimited number of projects and tests, online support, integration with Google-Analytics.

• Templates\editor: online editor.
• Demo version: 15 days.
• Comments: Convenient editor of your pages. There are no templates. Everything is beautiful, juicy and convenient, but from time to time it stops counting targets, errors / shoals of the service were detected, corrected by technical support, but the next day the pickup of targets again flew off for an unknown reason.

Service #6

• Service: http://www.clickthroo.com/
• Testing support: A/B.
• Tariff plans:

1. $195 , 50,000 visitors, 5 projects.
2. $395 , 100,000 visitors, 10 projects.
3. $695 , 100,000 visitors, unlimited number of projects.
4. $1195 , 250,000 visitors, unlimited number of projects.

• Templates\editor: templates and online editor.
• Demo version: 14 days demo access.
• Comments: You make a request for demo access by filling out a huge form and you don't get it. Ok, you get it, but after a couple of business days.

Service #7

$1295 - up to 10,000 visitors per month. Unlimited number of projects, pages, tests, domains, technical support.

Templates\editor: templates and online editor.

Demo version:free demo.

Development is one of the main components that are required to achieve a certain success. Any business needs development; without it, it will simply die, lose its relevance. The market is very volatile. Each time needs its own specific product. Due to the fact that the world does not stand still, every business needs to adapt to the audience, look for new, better ways of development.
Of course, first of all, an entrepreneur needs to make new unique selling propositions. They, in turn, must be of high quality and interesting in order to attract a large number of customers. For those who are afraid that changes will have a negative impact on work, there is an AB test.

AB testing is the practice of testing multiple changes to the same location to help you understand which changes will benefit the project the most.

This method will show the number of targeted actions, the time spent by users on your project page, it will also show the amount of revenue and the bounce rate.

Setup Guide:

Go to Google Analytics, category "Behavior", section "Experiments".Let me give you a simple example: you want to try changing the red button on the product page to blue. To test whether this is effective or not, you need to create two versions of the page. Give the name of the old version "A", the new version "B". Use Google Experiments to show visitors two different options, for example, over the course of a week.

Specify pages for testing. It is possible to specify additional options for the test.

• Set the experiment code to only HOME PAGE, you do not need to install the experiment code for option B. Whereas the standard Google Analytics code should be on both.

We paste the code on the site or send it to the programmer

• Testing is great for many tasks that involve changing the look of a web page. You can test any element on your site: different photos, different headlines, different content. Even just moving different elements around can have a dramatic effect on performance. Test the contact form on the right of the page against the same form on the left, and you could get twice as many messages sent from it. After placing the code, we will see the name of the experiment and the status “Running”:

Status "In Progress"

• When you click on it, we will see all the testing statistics:

Experiment statistics will be available by clicking on it

• Now, when visiting the tested page, users will see a link of the format:

By the way, if you want to see how the experiment works and whether it is configured correctly, go to the tested site from different browsers, Google will show option B after 3-5 attempts. This way you will make sure that the experiment is configured correctly.

Keep in mind that you will see slight differences and fluctuations even without changes to your own site. This is influenced by various factors, such as the season of the year, traffic sources, events, economics, competitor activities. Thus, if you tried testing during the week, it is better to repeat it next week or immediately set up the test for 2 weeks.

And don't forget that you can test different elements of your site at the same time, you don't have to finish one test before you start the next one.

AB testing— a great opportunity to improve the content of your project!
If you love the blue color, this does not mean the success of the blue buttons on the site)
Subscribe to a useful private newsletter

Dmitry Dementy

As you know, there are no static states in business. The company must constantly develop to meet the current market situation, the needs of customers and owners. Having stopped the development, the project immediately begins to degrade. For example, you cannot create an online store, add 200 products to the site and make a monthly profit of 100 thousand rubles. In order for the profitability of the project to at least not fall, the entrepreneur needs to constantly expand the assortment, increase the reach of the audience through advertising and the publication of useful content, improve the site's behavioral metrics and conversion rate.

One of the tools for developing web projects is A/B testing. This method allows you to measure audience preferences and influence key site performance indicators, including conversions, time spent on the page, average order value, bounce rate, and other metrics. In this article, you will learn how to properly conduct A/B testing.

What is A/B Testing

A/B testing is a marketing technique used to measure and manage the performance of a web page. This method is also called split testing (from the English split testing - separate testing).

A/B testing allows you to evaluate the quantitative performance of two versions of a web page, as well as compare them with each other. Split testing also helps evaluate the effectiveness of page changes, such as adding new design elements or calls to action. The practical meaning of using this method is to find and implement page components that increase its effectiveness. Pay attention again, A / B testing is an applied marketing method with which you can influence the conversion, stimulate sales and increase the profitability of a web project.

Split testing starts with evaluating the metrics of an existing web page (A, control page) and looking for ways to improve it. For example, you have created an online store. Imagine this store's landing page with a 2% conversion rate. The marketer wants to increase this figure to 4%, so he plans changes that will help solve this problem.

Let's say a specialist thinks that changing the color of a conversion button from neutral blue to aggressive red will make it more visible. To see if this results in more sales and more conversions, the marketer creates an improved version of the web page (B, new page).

With the help of split testing tools, the expert randomly splits the traffic between pages A and B into two approximately equal parts. Relatively speaking, half of the visitors land on page A, and the other half on page B. At the same time, the marketer keeps traffic sources in mind. To ensure the validity and objectivity of testing, it is necessary to send to pages A and B 50% of the visitors who came to the site from social networks, natural search, contextual advertising, etc.

After collecting enough information, the marketer evaluates the test results. As stated above, Page A's conversion rate is 2%. If it was 2.5% on Page B, then changing the conversion button from blue to red really increased the effectiveness of the landing. However, the conversion rate did not reach the desired 4%. Therefore, the marketer is further looking for ways to improve the page through A / B testing. In this case, the page with the red conversion button will act as a control page.

What to test

As noted above, split testing is an applied method that allows you to influence various website metrics. Therefore, the choice of the object of testing depends on the goals and objectives that the marketer sets for himself.

For example, if your landing page has a bounce rate of 99%, with most visitors leaving the landing page within 2-3 seconds of landing, you might want to consider changing the visual components of the page. With the help of an A/B test, a marketer can find the best page layout option, choose an attractive color scheme and images, and use a readable font. And if the marketer is faced with the task of increasing the number of subscriptions, he can try to change the corresponding conversion form. A split test will help a specialist choose the optimal button color, the best text option, the number of fields in the subscription form, or its location.

Most often, marketers test the following elements of web pages:

• The text and appearance of the conversion buttons, as well as their location.
• Title and description of the product.
• Dimensions, appearance and location of conversion forms.
• Page layout and design.
• The price of the product and other elements of the business proposal.
• Product images and other illustrations.
• The amount of text on a page.

What split testing tools to use

To perform A/B testing, a marketer needs to use one of the specialized services. The most popular of these is Google's Content Experiments, available to users of the Analytics system. Until mid-2012, this tool was called Google Website Optimizer. With it, you can test various page elements, including headings, fonts, conversion buttons and forms, images, and more. The Content Experiments service remains free, which is one of its main advantages. Its disadvantages include the need to work with HTML code.

You can also use the following Russian and foreign tools for split testing:

• Optimizely is the most popular paid A/B testing service on the internet. The cost of using it ranges from $19 to $399 depending on the type of subscription. The advantages of this service include the ability to create experiments in a visual interface, which saves the marketer from having to work with the HTML code of the pages being tested.
• RealRoi.ru is another domestic service that allows you to conduct A / B testing. Among the main advantages, one can single out that it is free and very easy to use. You can see how it works in detail in the following video:
• Visual Website Optimizer is a paid service that allows you to test various elements of a page. To use this tool, a marketer needs to be familiar with HTML code. Subscription prices range from $49 to $249.
• Unbounce is a service designed to create and optimize landing pages. In particular, it allows you to perform A / B testing. The cost of using it is from 50 to 500 dollars per month. Domestic analogue - LPGenerator. This service allows you to test only the pages created with it.

How to A/B Test with Content Experiments

The Google Analytics Experiments service allows you to simultaneously test the effectiveness of five variations of a page. Using it, marketers can perform A/B/N testing, which differs from standard A/B experiments by the ability to monitor the performance of several new pages, each of which can have several new elements.

The marketer has the ability to independently determine the proportion of traffic involved in testing. The minimum duration of the test is two weeks, the maximum is limited to three months. The specialist can receive data on the test results by e-mail.

To split test with Content Experiments, follow these steps:

1. Sign in to your Google Analytics account, select the site you want to test the performance of. After that, select the "Behavior - Experiments" menu.

1. Enter the URL of the page you will be testing in the appropriate form and click the "Start Experiment" button.

1. Choose a name and purpose for testing. Determine the percentage of traffic participating in the experiment. Decide if you want to receive email notifications of test progress. Click the "Next" button after selecting the required options.

1. Select the variations of the page to be tested. Add them to the appropriate forms and click Next.

1. Create an experiment code. If you don't know how to embed it on the page, select the "Send Code to Webmaster" option. If you don't break a sweat at the mention of HTML code, select the "Insert Code Manually" option.

Choose "Insert code manually" if you know how to handle HTML code

1. Copy the code marked in the previous illustration and paste it into the source code of the control page. The code must be inserted directly after the tag . After completing this action, click the "Save Changes" button.

1. Check for the test code on the control page and click the "Start Experiment" button. Note that the code only needs to be added to the control page.

You will be able to evaluate the first test results in a few days after the start of the experiment. To keep track of test results, select the appropriate experiment from the list and go to the reports page.

Ideas that should definitely be tested with split testing

As noted above, A/B testing helps increase the effectiveness of web pages. For this marketing method to bring results, the marketer must generate ideas that can positively influence certain site metrics. You can’t just take any changes from the ceiling, implement them and test the effectiveness. For example, site metrics are unlikely to change if you simply decide to change the page background from blue to light green.

A marketer needs to see ways to improve pages and understand why they should work. Split testing simply helps to test the specialist's assumptions. However, every marketer sometimes finds himself in a situation where all ideas have been tested, but the desired result has not been achieved. If you find yourself in this situation, try implementing the following changes and see if they work:

• Remove extra fields from the conversion form. Perhaps your potential subscribers do not want to disclose their passport details.
• Add the words “free” or free to the conversion page. Of course, the audience knows that subscription to the newsletter is free. But sometimes the word free works real miracles, because free vinegar is sweet.
• Post a video to your landing page. This usually has a positive effect on a number of metrics, including bounce rate, conversion rate, and time on page.
• Increase the period during which users can test your product for free. This is a simple and effective way to increase conversions for companies selling software and web services.
• Experiment with the color of the conversion buttons. In some cases, aggressive red buttons work well. However, sometimes they annoy users. Use an A/B test to find the most effective button color for your site.
• Promise bonuses to the first 10 or 100 buyers (subscribers). Do not rush to remove this promise even after the end of the promotion. Many users do not expect to be among the lucky ones, but still subconsciously react to a profitable offer.

How and why to test different variations of pages

Split testing allows you to evaluate the effectiveness of changes to web pages. This marketing method has applied value. It allows you to almost constantly improve pages by improving various metrics.

To test this or that change, you need to create a new version of the page and save the old one. Both options must have different URLs. After that, you should use one of the services for conducting split tests, for example, Content Experiments. Evaluation of test results can be carried out at least two weeks after the start of the experiment.

Do you think it's worth doing A/B testing? In what cases does this marketing method remain a waste of time?

kak-provodit-a-b-testirovanie

(split testing, A / B testing, Split testing) on ​​the site is a marketing method that consists in monitoring the control (A) and test (B) groups of elements - site pages that differ only in some indicators in order to increase the site conversion . Pages are shown to visitors alternately in equal shares, and after reaching the required number of impressions, the most conversion option is determined based on the data received.

Stages of A/B testing

In general, the entire A/B testing process can be summarized in 5 steps:

Step 1. Goal setting (business goals, conversion, website goals)

Step 2 Fixing initial statistical data

Step 3 Test setup and process

Step 4 Evaluation of results and implementation of the best option

Step 5 Repeat the experiment on other pages or with other elements as needed

Test duration

The duration of the experiment depends on the available traffic on the site. Conversion rate, as well as differences in the tested options. Many services automatically determine the duration. On average, 100 conversion actions on the site are enough and takes about 2-4 weeks.

Pages for testing

For testing, you can select any page of the site that is important in terms of conversion. Most often, this is the main page, registration / authorization pages, sales funnel pages. In this case, it is better to pay attention to the following points:

1. Most visited site pages
2. Pages with high visits
3. Refusal pages

The first is necessary for the purity of the experiment, the second and third to identify weaknesses on the site.

Most often, buttons, text, a slogan-call to action and the layout of the page as a whole are chosen for testing. To select an element, you can use the following algorithm of actions:

• A hypothesis is put forward about the behavior of the visitor
• A solution is proposed to change the elements (it is better to take 1-2, no more)

1. Add the word "Free"
2. Submit explainer video
3. Glue the registration button to the top of the page
4. Reduce the number of fields in the application
5. Add special offer counter
6. Add Free Trial
7. Change button colors or text on them

Test Automation

There are several paid and free tools for automating the testing process with a different set of features. A large list can be viewed. The most popular can be experiments in Google Analytics. It is free, Russified, easy to learn, and if a counter is installed on the site, then you do not need to wait for the collection of initial data and you can start the experiment in just a couple of clicks.

A/B testing with Google Analytics

Consider the process of creating a test in Google Analytics. To do this, go to the Reports->Behavior->Experiments tab. Enter the URL of the page you are testing and click "Start Experiment".

The next step is to fill in the fields: the name of the experiment, the goal (you can choose from the configured goals for the site), the coverage of site visitors for the experiment (it is better to set 100%).

In the second step, you will need to specify the addresses of the main (control) page and its variants.

If everything is done correctly, the system will give a green light to start testing.

The result of the experiment is very visual and may look like this:

Contrary to popular belief (after all, duplicate pages are created), such testing does not have a negative impact on the position of the site. It is enough to write rel="canonical" on alternative pages.

Important about A/B testing

1. Test versions of pages should not differ by more than 2 elements
2. Traffic between pages should be equally distributed
3. When making settings, select new site visitors
4. The results can only be judged by a wide sample, preferably at least 1000 people.
5. Evaluate results at the same time
6. Do not trust yourself, not all users think the way you do, so your preferred option may not be a winning one.
7. The results of A/B testing may not always bring the desired results in increasing conversions. So you need to experiment with other elements.

The design decision process has always been a popular topic of discussion. Why do some designers make choices that others wouldn't, and why do some designs seem to work better than others?

From academic research to sketches and anecdotes, the design world is passionate about process. The joke about Google and its 41 shades of blue lingered the longest.

Trying to determine which of the 2 shades of blue to use for link text, Google tested not only the main 2, but 39 in between. The story describes a roughly minute decision but highlights a rapidly growing approach to decision making. It is based on experiments, alternatives, and, most importantly, data.

"A/B testing can provide value, but it shouldn't come at the expense of other areas of design."

But why did Google test 41 shades of blue, and how can this approach help you or your organization? In this article, we will look at A/B testing (or multivariate testing): what it is, why it should be used, and its limitations.

A/B and multivariate testing in a nutshell

In short, A/B testing is a method of comparing 2 versions of something against each other to determine which version is better. The subject of testing can be an image, a button, a title, etc.

Multivariate testing is an extension of A/B testing where more than 2 versions are compared and (often) more variations are included. This will allow you to experience several elements at once and how they interact.

For simplicity, the rest of this article will only cover A/B testing, but for multivariate testing, the principles remain the same.

Why A/B Testing

The purpose of A/B testing is to allow you to make incremental improvements to your website or application. By contrasting your app or website with one or more variations, you can continually improve your design, get validation from real users.

In A/B testing, each test generates new data about what works and what doesn't. What works can be incorporated into a website or app and form a new and improved design.

A/B testing in the real world

To understand how A/B testing can be used and what it can do for you, you can check out hundreds of examples on websites like Which Test Won. You can also take a look at these popular studies:

Basic A/B testing process

Step 1: Where to test

To conduct A/B testing you will need a website or application. (A/B testing encourages the incremental improvement of an existing product and is not suitable for testing redesigns or new products and services.)

You have to decide which area of ​​your site or application you want to explore, and ideally, try to improve. The field of study can be selected based on various sources:

• Analytics: whether your analytics show that a particular page or screen is an annoyance to users. Are all your users exiting the same page?
• Usability testing: Did usability testing reveal a problem area or interaction? Have you tested a new solution and now want to test it on a large scale?
• Intuition or personal dislike: Do you believe something can be improved and want to test it with data? Is there something you've always disliked and where do you want to apply alternatives?

Most often, these three sources determine the subject of testing. Armed with this knowledge, you can move on to step 2.

Step 2: What to test (and what to measure)

One of the key aspects of A/B testing is that you only change one variable at a time. At first glance, this seems like a simple task, but you can quietly step over the mark and add more variables.

For example, if you want to test a button, you can try changing its text:

Or change color:

But if you combine both options and test the button with different text and different color, you would drastically reduce the value of the test.

Comparing these two buttons to each other, you would not be able to point out why they give different results: what kind of performance gain did the text change bring, or the color change.

“In order to conduct valuable A/B testing, it is critical to limit changes to a single variable.”

So, in order to conduct valuable A/B testing, it is essential to limit changes to a single variable. If you want to test multiple variables at the same time, you should do a multivariate test where you can test those multiple variables and get a better idea of ​​what effect each change had.

Whichever test you choose to run, you should also highlight the key metric you will be tracking. In the case of buttons, you will most likely measure the number of people who click on it. When it comes to title changes, you'll probably be looking at bounce rates, or time spent on the site.

What you track becomes what you test. Just make sure you know what you're trying to improve before you start A/B testing.

Step 3: How to Test

Now that you understand what and where you will be testing, let's talk about how we will do it. There are many applications for A/B testing. Here are the most popular:

• Google Analytics
• Optimizely
• Visual Website Optimizer
• A/B Tasty

These (and others) offer a basic A/B testing process, but differ in features. Your choice will depend on your developer skills, the flexibility you need, or simply the price.

Many large organizations often use more than one tool, depending on the amount of work required, or personal preference, so choosing the right tool will depend on your personal circumstances.

Step 4: How big is the test

So, you've agreed on the location of the test, the variables you'll be optimizing, and how you'll technically implement it all. The last question that remains to be answered is: how many users will be tested?

Some tools (such as Google Analytics) do not allow you to determine who will see the original version and who will see the alternative version, or even how long the test will last. For a beginner, this can be a useful feature as it simplifies the whole process.

"A/B testing can't tell you if you're solving the right problem."

If you want to set these variables yourself, then it's worth considering how long the test will last, and what percentage of the user will see the original version and what percentage of the alternative.

If you work in an organization that does not accept risks, then show the alternative to only 5-10% of users, and divide the rest 50:50. Ultimately, the choice will depend on your ambitions and the level and type of traffic your website or app receives.

When answering the question of how to divide the test, and how long should it take, ask yourself: how big should the test be so that I can be sure that the results are correct?

This is referred to by the technical term "statistical significance", or "statistical significance". Your goal is to create a test with a sample size large enough so that you can say with over 95% certainty, "Their change caused this result."

How you split the test is up to you, but how long it takes can depend on the amount of traffic the website or app receives. Don't worry, as intimidating as it sounds, there are plenty of online calculators to help you figure out if your results are statistically significant or if you need to increase your testing time.

• Visual Website Optimizer's significance calculator
• Kissmetrics value calculator

Step 5: Analyze and Decide

Here are the results! You ran the test, made sure it was statistically significant, and now you have the numbers.

Given all the work done, the number of people involved, everyone expects to see something like this result:

But more often than not, what you'll get is this:

Don't get frustrated (and don't despair) - A/B testing makes incremental improvements. And while big changes are possible, any improvement is a great start and puts you on the right track.

Even if the data shows you haven't made any improvements, you are now in a stronger position because you know what works and what doesn't.

"Data does not equal understanding"

If the test was successful, the next steps are up to you. You can submit a new version as soon as possible. Or, if you have done a small test, you can do another test with a lot of people.

What you do with the acquired information is ultimately up to you!

Understanding the Limitations of A/B Testing

As powerful as A/B testing is, it is also important to understand its shortcomings and limitations. Despite its growing popularity, A/B testing is not a panacea that can save any company, but rather another tool in your arsenal.

When considering the use of A/B testing, it is important to understand what it cannot do:

• Say why. A/B testing is a fantastic tool to help you understand what works and what doesn't. However, it cannot tell you why. To do this, you need to conduct high-quality user research. It is very important that you understand that data does not equal understanding.
• Help you test your app or website redesign. While in theory, you could compare the entire page design to the alternative and get success data, you wouldn't be able to figure out what it was about the design that was causing the difference. Until you do user research, the result will be meaningless.
• Tell you if you are solving the right problem. Because of the incremental nature of A/B testing, it can be a powerful tool to continually improve your website or application. A/B testing cannot tell you if you are solving the right problem. You can focus your tests on the home page and see improvements, but another part of the site may be the problem. This is a concept known as a local maximum.

What can A/B testing do for you?

If all this has whetted your appetite for A/B testing, then you probably already have the information you need to get started. Some small details may differ, or your organization may have specific requirements, but the general principles remain the same.

A/B testing, when used correctly, can be a great tool. It can allow your company to gradually improve and increase your success.

But it's important to understand that A/B testing is just one tool in any designer's much larger arsenal. In his resignation note, Doug Bowman, former head of visual design at Google, also recalled the anecdote about Google's 41 shades of blue. Thus, even though A/B testing can provide great value, it should not be at the expense of other areas of design.