Skip to main content

What is thumbnail A/B testing

Updated

What thumbnail A/B testing is, how YouTube's Test & compare picks a winner, what to change between versions and how to read Winner or Inconclusive.

Quick answer

Thumbnail A/B testing means showing different versions of a video's thumbnail to different viewers at the same time and keeping the one that performs best. On YouTube you do it with Test & compare in YouTube Studio: up to 3 versions, and the winner is the one with the highest watch time share, not the highest CTR.

Key takeaways

  • An A/B test runs thumbnail versions side by side on the same video, so the day, the season and the video's age affect them equally.
  • Test & compare in YouTube Studio tests up to 3 titles, thumbnails or both, from a computer, and finishes within two weeks.
  • The winner is the version with the highest watch time share, not the highest CTR. Results are Winner, Performed Same or Inconclusive.
  • Change one variable per test, make the difference large enough to see at phone size, and run tests on videos with steady traffic.

Thumbnail A/B testing means showing two or more versions of a video's thumbnail to different viewers at the same time and measuring which version performs better. Because the versions run side by side on the same video, the comparison is not skewed by the day of the week, a holiday or the age of the video. On YouTube you run these tests with Test & compare in YouTube Studio.

For a creator, it replaces guessing. Swapping a thumbnail by hand on Tuesday and comparing Wednesday's numbers with Monday's tells you very little, because the audience on those days is different. A test answers one question cleanly: for this video and this audience, which image earns more watching.

How YouTube Test & compare works

Want a thumbnail like this in 30 seconds?

Describe your video and let the AI design it. No Photoshop, no design skills.

Generate my thumbnail free
RuleWhat it means for you
Up to 3 versionsYou can test two or three options per video. Two is the cleanest comparison.
Title only, thumbnail only, or bothTest thumbnails alone if you want to learn about the image. Testing both mixes two variables.
Computer onlyTests are set up in YouTube Studio on a computer, not in the mobile app.
Some videos are not eligibleNot available for Shorts, scheduled live streams, Premieres, made-for-kids, mature or private videos.
Winner is the highest watch time shareThe version that leads to the most watching wins, even if another one got more clicks.
Finishes within two weeksPlan for up to two weeks per test. Do not judge it after a day.
Three possible resultsWinner, Performed Same or Inconclusive.

Warning

Test & compare does not pick the version with the highest CTR. A thumbnail that overpromises can win the click and still lose the test, because viewers leave early and the watch time share goes down.

How to set up a thumbnail test

  1. Make two or three versions as 16:9 JPG or PNG files, ideally 1280 x 720 or 3840 x 2160 pixels, that differ in one thing.
  2. Open YouTube Studio on a computer, go to Content and select the video.
  3. In the video's details, choose Test & compare and pick thumbnail only.
  4. Upload your versions and save.
  5. Leave the test alone until it finishes, then read the result and write down what you changed.

The best videos to test are new uploads you expect to get steady traffic, or older videos that still get views every week. A video that has stopped getting impressions has nothing to test with.

What to change between versions

Change one variable per test. If version B has different text, a different expression and a different background, a win tells you B is better but not why, and you cannot reuse the lesson on the next thumbnail. Keep everything else identical: same crop, same position, same font, same colors.

VariableVersion AVersion B
Text3 MISTAKES in yellow, top leftSTOP DOING THIS in yellow, top left
ExpressionCalm half smile, looking at cameraWide eyes, mouth open, looking at the object
Background colorDark navy behind the subjectBright yellow behind the subject
FramingTight crop, face filling the left halfWider shot showing the face and the whole setup
SubjectCreator's face with the productProduct alone, large, on a plain background
Text or noneTwo words of textSame image with no text

Example

A budget travel video. Version A: the creator on the right half, pointing at a hotel room behind, with the words 12 DOLLARS in white with a black outline at the top left, dark blue sky. Version B: identical, but the words read WORST HOTEL. Same crop, same face, same colors, so whatever the result, you learn whether price or verdict pulls more watching for this audience.

How many views a test needs

There is no fixed number, but the pattern is consistent. With little traffic, the result tends to come back Inconclusive, because a handful of views cannot separate real preference from chance. The smaller the difference between your versions, the more traffic it takes to tell them apart. A test between two clearly different thumbnails on a video with steady daily views has the best chance of producing a Winner. If your channel is small, test bigger differences and fewer versions: two options with a strong contrast give a result sooner than three options that look alike.

AI-powered

Turn this idea into a real thumbnail, right now

  • 9 click-ready designs in one go
  • No design skills required
  • Tuned to lift your click-through rate
Generate my thumbnail free

Free to start. No credit card. Cancel anytime.

How to read the result

  • Winner: one version earned clearly more watch time share. Keep it on the video, and note the variable you changed. That note is the real value of the test.
  • Performed Same: the versions earned a similar watch time share. The thing you changed did not matter much to this audience. Keep whichever you prefer and test a bigger change next time.
  • Inconclusive: there was not enough data to tell. This does not mean the versions are equal. It usually means the video did not get enough traffic during the test.

One test is one data point. If a face with a strong expression wins on three different videos, you have a pattern you can apply to every thumbnail. If it wins once and loses twice, it depends on the topic.

What a test does not tell you

A test only compares the versions you gave it. If all three thumbnails are weak, one of them still wins, and the winner is simply the least weak option. It also says nothing about versions you did not try, so a Performed Same between two text options does not mean text never matters on your channel. And the result belongs to that video's audience: a winner on a tutorial watched mostly through search may not carry over to a vlog shown mostly on the home feed. Before you test, check each version at 168 pixels wide and drop any that fail that check, so the test spends its traffic on options that could actually work.

Common A/B testing mistakes

  • Changing several things at once. You get a winner but no lesson you can reuse.
  • Stopping early or swapping a version mid-test because one looked ahead after a day. Early numbers swing, and the test needs its full run.
  • Testing near-identical variants, such as the same image with the text 5 pixels larger. Nobody notices the difference at 168 pixels wide, so the result comes back Performed Same or Inconclusive.
  • Testing on a video with almost no traffic and treating Inconclusive as a loss.
  • Judging by CTR instead of watch time share. The version with more clicks is not always the one that wins.
  • Testing title and thumbnail together when you only wanted to learn about the image.

If making a second and third version is the step that stops you testing, ThumBEAST can generate variants from a text description in about 30 seconds, which you can then adjust so only one variable differs.

Frequently asked questions

What is thumbnail A/B testing?

It is showing two or more versions of a video's thumbnail to different viewers at the same time and keeping the one that performs best. On YouTube, Test & compare in YouTube Studio does this for you.

How does YouTube Test & compare choose a winner?

It picks the version with the highest watch time share, not the highest click-through rate. A thumbnail that gets more clicks but loses viewers early can still lose.

How many thumbnails can I test on YouTube?

Up to 3. You can test titles only, thumbnails only, or both together.

How long does a thumbnail test take?

Tests finish within two weeks. Let the test complete instead of judging it from the first day's numbers.

Why is Test & compare not available for my video?

It is not available for Shorts, scheduled live streams, Premieres, made-for-kids, mature or private videos. It also has to be set up from YouTube Studio on a computer.

What does Inconclusive mean in a thumbnail test?

There was not enough data to separate the versions. It usually happens on videos with little traffic or when the versions are too similar. It does not mean they performed the same.

What should I change between thumbnail versions?

One variable at a time: the text, the expression, the background color or the framing. Keep the rest identical so the result tells you what made the difference.

AI-powered

Your next viral thumbnail is 30 seconds away

ThumBEAST's AI designs scroll-stopping YouTube thumbnails for you. Describe your video and get options you can publish today.

Generate my thumbnail free

Free to start. No credit card. Cancel anytime.