Case Study · AI / SaaS

UX Pilot Found a Feature That Was Already Converting Users. It Just Wasn't Turned On.

UX Pilot is an AI-powered UX/UI design platform that turns ideas into wireframes, user flows, and high-fidelity screens in minutes. Product analytics showed a huge conversion gap hiding in plain sight - so we ran the experiment to close it.

Role

Head of Product · 1st hire

Sector

AI · Design SaaS

Product

Wireframe & UI generation

Focus

Free-to-Paid Activation

Book a free intro call →

+44.67%

Free-to-Paid CVR lift from this Deep Design experiment (96.2% win probability, 120k users / 11 days)

$300k → $450k+

MRR growth across the full engagement

+50%

Free-to-Paid CVR lift across the full engagement (~3% → 4.5%+)

The Engagement

This case study is one example from a full-time Head of Product engagement. Aleksander joined UX Pilot as its first product hire, taking product ownership from the founder across a 1M-user AI tool - strategy, discovery, product analytics, experimentation, and the product operating model. The Deep Design experiment below is a single, representative slice of that work.

The Situation

The data already knew. The product just hadn't caught up.

UX Pilot has a feature called Deep Design - a higher-fidelity generation mode that produces noticeably more polished first drafts. It existed, but it wasn't on by default for new users. Digging into product analytics turned up a correlation too large to ignore.

Free-to-Paid conversion, first 7 days of use

1.3%

Didn't trigger
Deep Design

12.3%

Triggered
Deep Design

An 845% relative difference - before the product changed a single pixel.

Inside the Experiment

Toggle on Deep Design for first-time users

Test Card 17.10.2025 · 2-week test window · Owner: Aleksander Uznański

We believe that

Toggling on Deep Design by default will lead to an increase in Free-to-Paid activation, thanks to raising quality perception among first-time users.

What we built

Toggled on Deep Design by default for every user's first design generation, tested live against the existing default in an A/B split.

What we measured

Sign-up-to-generation CVR
Free-to-Paid Activation CVR

Test setup

120,000 users exposed over 11 days
Bayesian A/B test, 95% confidence

“Users who triggered Deep Design in their first 7 days converted to paid at 12.3%. Users who didn't: just 1.3%.”

What Happened

The correlation held up as causation

Learning Card 3–14.10.2025 · 120k users · 95% confidence

Result

Free-to-Paid conversion rose 44.67%, with a 96.2% win probability in the Bayesian model.

Insight

Exposing free users to Deep Design on their very first design generation measurably increases the likelihood they convert to paid.

Decision

Rolled out to 100% of users - Deep Design is now on by default for every first design generation.

Why This Matters For You

The best experiment might already be sitting in your analytics

Nobody had to guess at this hypothesis - the product analytics data all but wrote it for us. The actual work was noticing the pattern, turning it into a testable Test Card, and running it rigorously enough (120k users, a proper Bayesian model) to trust the result before shipping it to everyone. Most teams have a correlation like this sitting in their dashboards right now. Few have looked for it.