What is thumbnail A/B testing
What thumbnail A/B testing is, how YouTube's Test & compare picks a winner, what to change between versions and how to read Winner or Inconclusive.
Quick answer
Thumbnail A/B testing means showing different versions of a video's thumbnail to different viewers at the same time and keeping the one that performs best. On YouTube you do it with Test & compare in YouTube Studio: up to 3 versions, and the winner is the one with the highest watch time share, not the highest CTR.
Key takeaways
- An A/B test runs thumbnail versions side by side on the same video, so the day, the season and the video's age affect them equally.
- Test & compare in YouTube Studio tests up to 3 titles, thumbnails or both, from a computer, and finishes within two weeks.
- The winner is the version with the highest watch time share, not the highest CTR. Results are Winner, Performed Same or Inconclusive.
- Change one variable per test, make the difference large enough to see at phone size, and run tests on videos with steady traffic.
Thumbnail A/B testing means showing two or more versions of a video's thumbnail to different viewers at the same time and measuring which version performs better. Because the versions run side by side on the same video, the comparison is not skewed by the day of the week, a holiday or the age of the video. On YouTube you run these tests with Test & compare in YouTube Studio.
For a creator, it replaces guessing. Swapping a thumbnail by hand on Tuesday and comparing Wednesday's numbers with Monday's tells you very little, because the audience on those days is different. A test answers one question cleanly: for this video and this audience, which image earns more watching.
How YouTube Test & compare works
Want a thumbnail like this in 30 seconds?
Describe your video and let the AI design it. No Photoshop, no design skills.
| Rule | What it means for you |
|---|---|
| Up to 3 versions | You can test two or three options per video. Two is the cleanest comparison. |
| Title only, thumbnail only, or both | Test thumbnails alone if you want to learn about the image. Testing both mixes two variables. |
| Computer only | Tests are set up in YouTube Studio on a computer, not in the mobile app. |
| Some videos are not eligible | Not available for Shorts, scheduled live streams, Premieres, made-for-kids, mature or private videos. |
| Winner is the highest watch time share | The version that leads to the most watching wins, even if another one got more clicks. |
| Finishes within two weeks | Plan for up to two weeks per test. Do not judge it after a day. |
| Three possible results | Winner, Performed Same or Inconclusive. |
Warning
Test & compare does not pick the version with the highest CTR. A thumbnail that overpromises can win the click and still lose the test, because viewers leave early and the watch time share goes down.
How to set up a thumbnail test
- Make two or three versions as 16:9 JPG or PNG files, ideally 1280 x 720 or 3840 x 2160 pixels, that differ in one thing.
- Open YouTube Studio on a computer, go to Content and select the video.
- In the video's details, choose Test & compare and pick thumbnail only.
- Upload your versions and save.
- Leave the test alone until it finishes, then read the result and write down what you changed.
The best videos to test are new uploads you expect to get steady traffic, or older videos that still get views every week. A video that has stopped getting impressions has nothing to test with.
What to change between versions
Change one variable per test. If version B has different text, a different expression and a different background, a win tells you B is better but not why, and you cannot reuse the lesson on the next thumbnail. Keep everything else identical: same crop, same position, same font, same colors.
| Variable | Version A | Version B |
|---|---|---|
| Text | 3 MISTAKES in yellow, top left | STOP DOING THIS in yellow, top left |
| Expression | Calm half smile, looking at camera | Wide eyes, mouth open, looking at the object |
| Background color | Dark navy behind the subject | Bright yellow behind the subject |
| Framing | Tight crop, face filling the left half | Wider shot showing the face and the whole setup |
| Subject | Creator's face with the product | Product alone, large, on a plain background |
| Text or none | Two words of text | Same image with no text |
Example
A budget travel video. Version A: the creator on the right half, pointing at a hotel room behind, with the words 12 DOLLARS in white with a black outline at the top left, dark blue sky. Version B: identical, but the words read WORST HOTEL. Same crop, same face, same colors, so whatever the result, you learn whether price or verdict pulls more watching for this audience.
How many views a test needs
There is no fixed number, but the pattern is consistent. With little traffic, the result tends to come back Inconclusive, because a handful of views cannot separate real preference from chance. The smaller the difference between your versions, the more traffic it takes to tell them apart. A test between two clearly different thumbnails on a video with steady daily views has the best chance of producing a Winner. If your channel is small, test bigger differences and fewer versions: two options with a strong contrast give a result sooner than three options that look alike.
Turn this idea into a real thumbnail, right now
- 9 click-ready designs in one go
- No design skills required
- Tuned to lift your click-through rate
Free to start. No credit card. Cancel anytime.
How to read the result
- Winner: one version earned clearly more watch time share. Keep it on the video, and note the variable you changed. That note is the real value of the test.
- Performed Same: the versions earned a similar watch time share. The thing you changed did not matter much to this audience. Keep whichever you prefer and test a bigger change next time.
- Inconclusive: there was not enough data to tell. This does not mean the versions are equal. It usually means the video did not get enough traffic during the test.
One test is one data point. If a face with a strong expression wins on three different videos, you have a pattern you can apply to every thumbnail. If it wins once and loses twice, it depends on the topic.
What a test does not tell you
A test only compares the versions you gave it. If all three thumbnails are weak, one of them still wins, and the winner is simply the least weak option. It also says nothing about versions you did not try, so a Performed Same between two text options does not mean text never matters on your channel. And the result belongs to that video's audience: a winner on a tutorial watched mostly through search may not carry over to a vlog shown mostly on the home feed. Before you test, check each version at 168 pixels wide and drop any that fail that check, so the test spends its traffic on options that could actually work.
Common A/B testing mistakes
- Changing several things at once. You get a winner but no lesson you can reuse.
- Stopping early or swapping a version mid-test because one looked ahead after a day. Early numbers swing, and the test needs its full run.
- Testing near-identical variants, such as the same image with the text 5 pixels larger. Nobody notices the difference at 168 pixels wide, so the result comes back Performed Same or Inconclusive.
- Testing on a video with almost no traffic and treating Inconclusive as a loss.
- Judging by CTR instead of watch time share. The version with more clicks is not always the one that wins.
- Testing title and thumbnail together when you only wanted to learn about the image.
If making a second and third version is the step that stops you testing, ThumBEAST can generate variants from a text description in about 30 seconds, which you can then adjust so only one variable differs.
Frequently asked questions
What is thumbnail A/B testing?
It is showing two or more versions of a video's thumbnail to different viewers at the same time and keeping the one that performs best. On YouTube, Test & compare in YouTube Studio does this for you.
How does YouTube Test & compare choose a winner?
It picks the version with the highest watch time share, not the highest click-through rate. A thumbnail that gets more clicks but loses viewers early can still lose.
How many thumbnails can I test on YouTube?
Up to 3. You can test titles only, thumbnails only, or both together.
How long does a thumbnail test take?
Tests finish within two weeks. Let the test complete instead of judging it from the first day's numbers.
Why is Test & compare not available for my video?
It is not available for Shorts, scheduled live streams, Premieres, made-for-kids, mature or private videos. It also has to be set up from YouTube Studio on a computer.
What does Inconclusive mean in a thumbnail test?
There was not enough data to separate the versions. It usually happens on videos with little traffic or when the versions are too similar. It does not mean they performed the same.
What should I change between thumbnail versions?
One variable at a time: the text, the expression, the background color or the framing. Keep the rest identical so the result tells you what made the difference.
Your next viral thumbnail is 30 seconds away
ThumBEAST's AI designs scroll-stopping YouTube thumbnails for you. Describe your video and get options you can publish today.
Generate my thumbnail freeFree to start. No credit card. Cancel anytime.
Try it on your own thumbnail
Free tools, no sign-up. They run in your browser.
Thumbnail Text Generator
Try your title in 15 thumbnail fonts
Thumbnail Preview
See your thumbnail inside YouTube before you publish
Thumbnail Downloader
Grab any video thumbnail in full resolution
Thumbnail Resizer
Fit any image to 1280x720 and under 2 MB
CTR Calculator
Work out your click-through rate and what one more point is worth
Title Generator
10 title ideas from your video topic