Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Purpose

Fake door testing is an easy yet very powerful method to measure interest in a product or service in a data-driven way without actually coding and or implementing the product or service itself (you can think of it as a MVP before the MVP).

We build a landing page as a so called "fake door" for our product/service and track how many people tried to enter this door (in other words: how many people want to buy, use or participate in the described product or service). 

We use fake door experiments mainly for two reasons in our Business Design process:

+ Exploration: Fake door experiments can help you to explore certain parts of your business model (e.g. job(s) to get done, target groups...).

+ Validation: Fake door experiments can help you to validate key parts of your business model (e.g. advertising channels, pricing...).

Info
titleNote

You also can use also use fake experiments in your scoping phase. If you have issues developing a "laser-focused" Project Charter, especially if you have uncertainties around the solution space you are exploring, fake door experiments can help you narrow down what is the most valuable area to focus on.

How to Build a Fake Door Experiment

The process of building and running a Fake-Door fake door Experiment consists of four steps:

  1. Design your new business: We always need a completed Business Model template before we start building anything.
  2. Define your uncertainties: Also the Hypothese & Experiments template is a must-have requirement for every fake door experiment.
  3. Build a landing page: We create a landing page which describes our offerings including our value proposition (see 1.) and targets 1-X questions / hypotheses (see 2.).
  4. Drive traffic to your landing page: We launch paid advertisements (see Online Ad Campaigns) or use other channels to bring users to the landing page.
  5. Track and analyze user behavior: We measure everything our users do on our landing page (via Google Ads, Google Analytics and custom tracking). Ans And finally, we analyze the user behavior and benchmark it against our assumptions with dashboards.


How to Run a Fake Door Experiment

To generate the most insights, fake door experiments are best executed along the following process model:

  1. --- Run the experiment for one week ---
  2. Analyze: Review the data and have a close look on what you've learned 
  3. Adjust: Based on 2., adjust your campaigns (optimize keywords, click-through-rates, conversion-rates, fine-tune target groups etc.) and the tracking  
  4. --- Repeat until we don't learn anymore ---

Need Support?

Markus Sorg

We build fake door experiments in a couple of days!
Contact us: support@orangehills.com.


Instructions for Coaches

  1. With fake door experiments you can generate a lot of data. And because it's easy to generate them, we offen fall into the trap: We need to be 100% clear about the insights we want to generate before we start the experiment. Make sure your team is crystal clear about what they want to learn and why.
  2. Make sure you have an agile approach when running fake door experiments: Run the experiment for one week, pause, look at the results, benchmark against your desired learnings, adjust your campaigns and tracking. Do this in weekly cycles until you learned enough.
  3. Be aware that fake door experiments generate "soft signals". We can see HOW users behave but not WHY they behave in a certain way. If you need to know why people behave like they do, you need to do interviews or observe them.

Q & A

  • How can I build meaningful dashboards and integrate all these different data sources (Google-/Facebbok-/Twitter-Ads, Google Analytics, custom data)? Have a look at Google Data Studio. There is no coding necessary and you can integrate many data sources out of the box. 
  • How can I track custom user behavior on our site (Downloads, Configurator-Actions, Scroll-Positions etc) within Google Analytics? Have a look at Google Analytics Custom Events.

Insights

With fake door testing we will can generate useful insights. Let's have a look at some examples and how we can explore them:


Which insights do we want to generate?How do we test this?Link to Business Model

Exploration


What do people actually search for (what are their pains/job(s) to get done)?When running campaigns, we can analyze what users searched for on Google Search, Facebook etc.See "Offerings", "Job(s) to get done" and "Pains" in our Business Model
Which target groups convert better or worse?When running campaigns, we focus on different target groups based on their behavioral data (e.g. what they liked, which people/companies they follow, what they are interested in)  and measure the conversion rates.See "Target groups" in our Business Model

How much are the customer acquisition costs?

When running campaigns, we can exactly measure what the costs of a converted user are (CPC - Cost Per Click).See "Profit formula" in our Business Model
Where exactly in the description of our offering do users leave the page?We track scrolling positions on the landing page and measure jump-off points. Therefore we can analyze which part of the description of our business model is not yet convincing enough.All parts of our Business Model

Validation


Which advertising channels do convert better or worse?We run paid advertising campaigns on multiple different channels (e.g. Google, Youtube, Facebook, LinkedIn, Twitter, Forums or even offline channels like handouts etc.) and measure the conversion rates.See "Channels" in our Business Model
How does our story/message convert?We A/B-Test different messages/story elements in our campaigns and measure the conversion rates.See "Brand & messages" in our Business Model
Which keywords convert best?We optimize our campaigns for different keywords and measure the conversion rates on a keyword level.See "Brand & messages" in our Business Model
What is a pricing model which works for my target group?We A/B-Test different pricing models and measure the conversion rates.See "Profit formula" in our Business Model

Data Analysis 

Make sure you analyze the user behavior on your landing page properly and present the insights you generated in a very clear and understandable way. Map your data to the questions and hypotheses you have in a custom dashboard. Key metrics are: Users on the site, collected emails, impressions, click-trough-rates, cost per click, conversion rates etc.