Lessons Learned From 2,345,864 Website Popup Visitors

Again in 2015, Unbounce launched its first ever exit popup on this very weblog.

Did it ship our signup fee skyrocketing 4,000%? Nope.

Did it flip our weblog right into a conversion manufacturing facility for brand spanking new leads? Not even shut — our preliminary conversion fee was barely over 1.25%.

However what it did do was begin us down the trail of exploring one of the best methods to make use of this expertise; of furthering our objectives by discovering methods to supply guests related, worthwhile content material via popups.

On this put up, we’ll break down all of the wins, losses and “holy smokes!” moments from our first 2,345,864 exit popup viewers.

Objectives, instruments and testing circumstances

Our objective for this mission was easy: Get extra individuals to devour extra Unbounce content material — whether or not or not it’s weblog posts, ebooks, movies, you identify it.

We make investments lots in our content material, and we would like it learn by as many entrepreneurs as doable. All our analysis — every thing we find out about that elusive factor known as conversion, exists in our content material.

Our content material additionally permits readers to search out out whether or not Unbounce is a software that may assist them. We wish extra prospects, however provided that they’ll actually profit from our product. Those that expertise ‘lightbulb’ moments when studying our content material undoubtedly match the invoice.

Testing circumstances have been as follows:

1. All popups have been triggered on exit; which means they launched solely when abandoning guests have been detected.

2. For the primary three experiments, we in contrast sequential durations to measure outcomes. For the ultimate two, we ran makeshift A/B assessments.

3. When evaluating sequential durations, testing circumstances have been remoted by excluding new weblog posts from exhibiting any popups.

4. A “conversion” was outlined as both a accomplished kind (lead gen popup) or a click on (clickthrough popup).

5. All experiments have been performed between January 2015 and November 2016.

Experiment #1: Content material Provide vs. Generic Signup

Our first exit popup had a easy objective: Get extra weblog subscribers. It appeared like this.

It was considered by 558,488 distinctive guests over 170 days, 1.27% of which transformed to new weblog subscribers. Respectable begin, however not adequate.

To enhance the conversion fee, we posed the next.

HYPOTHESIS
As a result of on-line advertising affords usually convert higher when a particular, tangible provide is made (versus a generic signup), we anticipate that by providing a free e-book to abandoning guests, we’ll enhance our conversion fee past the present 1.27% baseline.

Whereas the unique popup requested guests to subscribe to the weblog for “suggestions”, the challenger popup provided guests The 23 Principles of Attention-Driven Design.

After 96 days and over 260,000 guests, we had sufficient conversions to name this experiment a hit. The popup transformed at 2.65%, and captured 7,126 new weblog subscribers.

Since we didn’t A/B take a look at these popups, our outcomes have been merely observations. Seasonality is certainly one of many components that may sway the numbers.

We couldn’t take it as gospel, however we have been seeing double the subscribers we had beforehand.

Observations

• Providing tangible assets (versus non-specific guarantees, like a weblog signup) can positively have an effect on conversion charges.

Experiment #2: 4-field vs. Single-field Popups

Information individuals at all times spoil the occasion.

The early success of our first experiment caught the eye of Judi, our resident advertising automation whiz, who correctly reminded us that gathering solely an electronic mail tackle on a large-scale marketing campaign was a missed alternative.

For us to completely leverage this marketing campaign, we would have liked to search out out extra concerning the people (and organizations) who have been consuming our content material.

Translation: We would have liked so as to add three extra kind fields to the popup.

Since filling out types is a common bummer, we safely assumed our conversion fee would take a dive.

However one thing else occurred that we didn’t predict. Discover a distinction (apart from the shape fields) between the 2 popups above? Yup, the brand new model was bigger: 900x700px vs. 750x450px.

Including three kind fields made our authentic 750x450px design really feel too cramped, so we arbitrarily elevated the dimensions — by no means pondering there could also be penalties. Extra on that later.

Anyhow, we launched the brand new model, and as anticipated the outcomes sucked.

Issues weren’t trying good after 30 days.

For enterprise causes, we determined to finish the take a look at after 30 days, although we didn’t run the challenger popup for an equal time interval (96 days).

Total, the conversion fee for the 30-day interval was 48% decrease than the earlier 96-day interval. I knew it was for good purpose: Constructing our knowledge warehouse is vital. Nonetheless, a small a part of me died that day.

Then it acquired worse.

It occurred to us that for a 30-day interval, that pattern measurement of viewers for the brand new popup (53,460) appeared awfully small.

A better inspection revealed that our earlier popup averaged 2,792 views per day, whereas this new model was averaging 1,782. So principally our 48% conversion drop was served a la carte with a 36% plunge in general views. Enjoyable!

However why?

It seems growing the dimensions of the popup wasn’t so innocent. The scale was too massive for many individuals’s browser home windows, so the popup solely fired two out of each three visits, even when focusing on guidelines matched.

We conceded, and redesigned the popup in 800x500px format.

Day by day views rose again to their regular numbers, and our new baseline conversion fee of 1.25% remained principally unchanged.

Massive hole between “masses” and “views” on June 4th; narrower hole on June fifth.

Observations

• Rising the variety of kind fields in popups may cause friction that reduces conversion charges.
• Popup sizes exceeding 800×500 might be too massive for some browsers and cut back load:view ratio (and general impressions).

Experiment #3: One Popup vs. 10 Popups

It appeared like such a fantastic thought on the time…

Why not get hyper related and construct a unique exit popup to every of our weblog classes?

With our new baseline conversion fee decreased to 1.25%, we would have liked an enchancment that might assist us overcome “kind friction” and get us again to that wholesome 2%+ vary we loved earlier than.

So with little supporting knowledge, we hypothesized that growing “relevance” was the magic bullet we would have liked. It works on landing pages — why not popups?

HYPOTHESIS  
Since “relevance” is essential to driving conversions, we anticipate that by operating a singular exit popup on every of our weblog classes — whereby the free useful resource is restricted to the class — we’ll enhance our conversion fee past the present 1.25% baseline.

We divide our weblog into classes in keeping with the advertising matter they cowl (e.g., touchdown pages, copywriting, design, UX, conversion optimization). Every put up is tagged by class.

So to extend relevance, we created a complete of 10 exit popups (every providing a unique useful resource) and assigned every popup to at least one or two classes, like this:

Creating all the brand new popups would take round three hours, however since we already had a deep backlog of assets on all issues on-line advertising, discovering a related e-book, course or video to supply in every class wasn’t tough.

And since our URLs comprise class tags (e.g., all posts on “design” begin with root area unbounce.com/design), ensuring the correct popup ran on the correct put up was straightforward.

URL Concentrating on rule for our Design class; the “embrace” rule routinely excludes the popup from operating in different classes.

However there was an issue: We’d established a strict rule that our readers would solely ever see one exit popup… regardless of what number of weblog classes they browsed. It’s a part of our philosophy on using popups in a way that respects the user experience.

Once we have been simply utilizing one popup, that was straightforward — a easy “Frequency” setting was all we would have liked.

…however not really easy with 10 popups operating on the identical weblog.

We would have liked a strategy to exclude anybody who noticed one popup from seeing any of the opposite 9.

Cookies have been the apparent reply, so we requested our builders to construct a brief answer that might:

• Cross a cookie from an popup to the customer’s browser
• Exclude that cookie in our focusing on settings

They obliged.

We used “incognito mode” to repeatedly take a look at the performance, and after that we have been go for launch.

Then this occurred.

Ignore the structure… the dashboard is far prettier now 🙂

After 10 days of information, our conversion fee was a mixed 1.36%, 8.8% larger than the baseline. It will definitely crept its strategy to 1.42% after an extra 250,000 views. Nonetheless nowhere close to what we’d hoped.

So what went flawed?

We surmised that simply because a suggestion is “related” doesn’t imply it’s compelling. Admittedly, not all the 10 assets have been on par with The 23 Principles of Attention-Driven Design, the e-book we initially provided in all classes.

That mentioned, this experiment offered an surprising profit: we may now see our conversion charges by class as an alternative of only one large quantity for the entire weblog. This may serve us nicely on future assessments.

Observations

• Simply because a suggestion is related doesn’t imply it’s good.
• Conversion charges differ significantly between classes.

Experiment #4: Useful resource vs. Useful resource

“Simply because it’s related doesn’t imply it’s good.”

This lesson impressed a easy goal for our subsequent activity: Enhance the affords in our underperforming classes.

We determined to check new affords throughout 5 classes that had low conversion charges and excessive visitors quantity:

1. A/B Testing and CRO (0.57%)
2. Electronic mail (1.24%)
3. Lead Gen and Content material Advertising and marketing (0.55%)

Notice: We used the identical popup for the A/B Testing and CRO classes, in addition to the Lead Gen and Content material Advertising and marketing Classes.

Speculation
Since we consider the assets we’re providing within the classes of A/B testing, CRO, Electronic mail, Lead Gen and Content material Advertising and marketing are much less compelling than assets we provide in different classes, we anticipate to see elevated conversion charges once we take a look at new assets in these classes.

With earlier research talked about on this put up, we in contrast sequential durations. For this one, we took issues a step additional and jury-rigged an A/B testing system collectively utilizing Visible Web site Optimizer and two Unbounce accounts.

And after discovering what we believed to be extra compelling assets to supply, the brand new take a look at was launched.

We noticed barely improved leads to the A/B Testing and CRO classes, though not important. For the Electronic mail class, we noticed a big drop-off.

Within the Lead Gen and Content material Advertising and marketing classes nonetheless, there was a dramatic uptick in conversions and the outcomes have been statistically important. Progress!

Observations

• Not all content material is created equal; some assets are extra fascinating to our viewers.

Experiment #5: Clickthrough vs. Lead Gen Popups

Though progress was made in our earlier take a look at, we nonetheless hadn’t solved the issue from our second experiment.

Whereas having the 4 fields made every conversion extra worthwhile to us, it nonetheless decreased our conversion fee a relative 48% (from 2.65% to 1.25% again in experiment #2).

We’d now labored our manner as much as a baseline of 1.75%, however nonetheless wanted a technique for decreasing kind friction.

The reply lay in a brand new tactic for utilizing popups that we dubbed visitors shaping.

Site visitors Shaping: Utilizing clickthrough popups to incentivize guests to maneuver from low-converting to high-converting pages.

Right here’s a fast illustration:

Changing to this format would require us to:

1. Redesign our exit popups
2. Construct a devoted touchdown web page for every popup
3. Acquire leads by way of the touchdown pages

Mainly, we’d be utilizing the popups as a bridge to maneuver readers from “ungated” content material (a weblog put up) to “gated” content material (a free video that required a kind submission to view). Kinda like enjoying ‘kind area sizzling potato’ in a modern-day model of Pipe Dream.

Speculation
As a result of “kind friction” reduces conversions, we anticipate that eradicating kind fields from our popups will enhance engagement (sufficient to offset the drop off we anticipate from including an additional step). To do that, we’ll redesign our popups to clickthrough (no fields), create a devoted touchdown web page for every popup and add the four-field kind to the touchdown web page. We’ll measure leads to Unbounce.

By this level, we have been utilizing the Unbounce Builder to create your entire marketing campaign: the clickthrough popups have been in-built Unbounce utilizing drag and drop, and the touchdown pages have been copied over (and up to date) from current campaigns (that already provided the assets)

We determined to check this out in our A/B Testing and CRO in addition to Lead Gen and Content material Advertising and marketing classes.

After filling out the shape, guests would both be given a safe hyperlink for obtain (PDF) or taken to a useful resource web page the place their video would play.

Once more, for this to achieve success the conversion fee on the popups would wish to extend sufficient to offset the drop off we anticipated by including the additional touchdown web page step.

These have been our outcomes after 21 days.

Not surprisingly, engagement with the popups elevated considerably. I stress the phrase “engagement” and never “conversion,” as a result of our objective had modified from a kind submission to a clickthrough.

To be able to see a conversion enhance, we would have liked to issue within the proportion of holiday makers who would drop off as soon as they reached the touchdown web page.

A fast examine in Unbounce confirmed us touchdown web page drop-off charges of 57.7% (A/B Testing/CRO) and 25.33% (Lead Gen/Content material Advertising and marketing). Time for some grade 6 math…

Even with important drop-off within the touchdown web page step, general internet leads nonetheless elevated.

Our subsequent step can be making use of the identical format to all weblog classes, after which measuring general outcomes.

Onward!

All observations

• Providing particular, tangible assets (vs. non-specific guarantees) can positively have an effect on conversion charges.
• Rising the variety of kind fields in popups may cause friction that reduces conversion charges.
• Popup sizes exceeding 800×500 might be too massive for some browsers and cut back load:view ratio (and general impressions).
• Simply because a suggestion is related doesn’t imply it’s good
• Conversion charges differ significantly between weblog classes
• Not all content material is created equal; some assets are extra fascinating to our viewers.
• “Type friction” can differ considerably relying on the place your kind fields seem.

Keep tuned…

We’re persevering with to check new triggers and focusing on choices for popups, and we need to let you know all about it.

So what’s in retailer for subsequent time?

1. The Set off Take a look at — What occurs once we take a look at our “on exit” set off towards a 15-second time delay?
2. The Referral Take a look at — What occurs once we present totally different popups to customers from totally different visitors sources (e.g., social vs. natural)?
3. New v.s. Returning Guests — Do returning weblog guests convert higher than first-time guests?