Hello, I'm non-sta representative Takasaki.
Our company believes that the success of an e-commerce site depends on “who, what and how”.
Specifically,

• to who, what kind of person or target
• What is the strength of any product or service?
• how do you tell them?

It is important to combine the above vectors.

However, I think it is hard enough to match all vectors perfectly at once with a lottery.

Therefore, we conduct an A/B test using a verification page and repeat hypotheses and verifications to perform tune-ups on who, what and how to convey them, contributing to the growth of our customers.

What is an AB test?

AB testing (A/B testing) is an experimental method used in marketing and web design.
Get two different versions of the page (A and B), show them randomly to users, and then make decisions based on data and maximize their effectiveness by comparing which results are better.

Main characteristics

• difference between A and B : Normally, A is the current one (control), and B is a new idea or change (test target) such as changing button colors on websites, changing email subject matter, adjusting ad language.
• random allocation : The user is randomly assigned to either A or B, so that bias does not occur.
• measurement indicator : Based on specific metrics such as click rate (CTR), conversion rate, sales etc.

Example

If you want to test the "Buy" button on your website:

• A: The red button (traditional design)
• B: Blue button (new design) Show A to half of users and B to the other half, and analyze which leads to more clicks or purchases.

merit

• It can be judged by objective data.
• You can see how the user actually reacts.
• You can try out improvement measures with less risk.

point of caution

• If the sample size is small, results may not be reliable.
• If the test period is too short, it can overlook seasonal factors and temporary effects.
• When multiple elements are changed at the same time, it becomes difficult to understand which changes have affected the outcome (in that case, consider a “multivariate test”).

Simply put, AB testing is a tool for scientifically verifying which one should be better.

Barack Obama Says AB Test Is a Success

In Barack Obama’s 2008 presidential campaign, A/B testing was a big one to optimize the messages and designs that popped up on the top page.

The test was aimed at converting website visitors to email magazine subscribers and even to donors or volunteers, which had a significant impact on fundraising and audience growth.

Here is a detailed description of the specific tests and their outcomes.

A/B Test Overview

purpose

Users who visit the website’s top page will be prompted to sign up for an email magazine, which will help them make donations and participate in volunteer activities.

test target

The Obama campaign's web team tested two key elements:

*Screenshots on the top page of a real website
https://blog.optimizely.com/2010/11/29/how-obama-raised-60-million-by-running-a-simple-experiment/ more citation

registration button wording

We tested 4 variations.
Sign Up," "Learn More," "Join Us Now" and "Sign Up Now."

Testing Main Visual and Messages on Top Page

We tested six versions (three for still images, three for video).

https://blog.optimizely.com/2010/11/29/how-obama-raised-60-million-by-running-a-simple-experiment/ more citation

Three Videos: Obama Speech, Supporters Voice and Campaign

4 buttons × 6 media types = A total of 24 patterns were tested, 310,382 visitors and each variation was seen by about 13,000.
We looked for the highest percentage of visitors registered in an email magazine (registration number the number of visitors who saw that variation).

hypothesis

The intuition of the staff was that dynamic messages using videos would attract users’ attention and increase their registration rate.
On the other hand, simple and familiar still images or soft registration buttons (e. g. “Learn More”) that do not prompt specific actions were unknown.

Test results: 40.6% better registration and $60 million more donations

As a result of this test, the combination of “Learn More” buttons and family photos is the most effective, reaching 11.6% of registration rates—a 40.6% improvement over 8.26% on original pages, with an additional 2.8 million registered and $60 million in donations.

Source
Optimizely: 「How Obama Raised $60 Million by Running a Simple Experiment」
Statsig: 「A/B testing and experimentation in the Obama 2008 and 2012 campaigns」
Medium: 「The Secret Behind Obama’s 2012 Election Success: A/B Testing」

In this way, based on the fact that AB test is not an individual's intuition but as a number, you can know what kind of things people who are looking at websites feel attractive and you can improve the web without relying on experience, instinct or courage.

How to Run AB Testing on Non-Star Flow Shopify

When doing A/B testing at Shopify, the tools we use vary depending on what we want to investigate but we do this to verify our hypothesis of who, what and how to communicate. Shougun A special site (LP) is created using an app called “A/B test”, and the reaction obtained there is often reflected on the main body site.

What is Shogun?

Shogun is an app available on the Shopify App Store that has been used by more than 35,000 brands since it was released in 2015, and is valued as a tool to enhance the design and optimization of the Shopify store.. Main purpose is for store owners to easily build pages using visual editors, and increase sales with A/B testing or personalization features.

cost

In addition to the price of Shopify itself, if you are going to do A/B testing with Shogun, you must subscribe to Shogun’s paid plan.

2,000 sessions/month: $39 per month
10,000 sessions/month: $249 per month

In addition, if you guide LP with a paid advertisement, the advertising cost will be charged.

Lastly: Improving the E-Commerce Site Scientifically

I think that the design and content of an e-commerce site is often determined by the intuition of a creator or decision maker, but as Barack Obama’s example above shows, the option against the team’s intuition may be correct for users.
In order to make a website that grows healthy, we believe that using A/B testing to create what users really want will be an e-commerce operation where the staff working at the same time is not exhausted.