Confessions Of A Lean Startup: How I Invalidated My Assumptions To Build A Better Product

Google+ Pinterest LinkedIn Tumblr +
In a previous blog post (under the Status Quo heading) I summarised the process most founders take from ideation to capital raising. To expand on what I wrote in the Status Quo paragraph, this is typically what I’ve seen in the startup ecosystem. Person A has been in an industry for a while, is disgruntled with how the technology or industry is currently operating and comes up with an amazing new idea that no one else is doing and thinks they can do it better, quicker, cheaper with a new product or service.The critical part here is they think the new product or service will provide greater benefits or better solutions to challenges their potential customers face. This is a dangerous and fundamental fallacy which often sounds something like “I’ve been in industry X for Y years and know my customers like the back of my hand. I’m going to spend $Z and 6 months developing my product in “stealth mode” and when my new product/service is ready I’m going to bring it to market and people are going to buy it like hotcakes.”With that said, the question them becomes “does person A really know, I mean REALLY know if their new product/service will fix the pain point of a particular customer segment?” The answer to that question is, they can’t know because they are doing one of a few things. They are either applying something new to an existing market, something that already exists to a different market or something new to a new market. In short, it’s never been tried before so how could person A know the outcome with any degree of certainty? All they have are assumptions… and therein lies the key.
[/vc_column_text][/vc_column][/vc_row][vc_row row_type=”full-width-content” unlock_row=”” row_height_percent=”0″ override_padding=”yes” h_padding=”2″ top_padding=”3″ bottom_padding=”3″ overlay_alpha=”100″ gutter_size=”3″ shift_y=”0″ row_height_use_pixel=””][vc_column column_width_use_pixel=”yes” overlay_alpha=”50″ gutter_size=”3″ medium_width=”0″ shift_x=”0″ shift_y=”0″ zoom_width=”0″ zoom_height=”0″ column_width_pixel=”800″][/vc_column][/vc_row][vc_row row_type=”full-width-content” unlock_row=”” row_height_percent=”0″ override_padding=”yes” h_padding=”2″ top_padding=”0″ bottom_padding=”3″ overlay_alpha=”100″ gutter_size=”3″ shift_y=”0″ row_height_use_pixel=””][vc_column width=”1/6″ column_width_use_pixel=”yes” overlay_alpha=”50″ gutter_size=”3″ medium_width=”0″ shift_x=”0″ shift_y=”0″ zoom_width=”0″ zoom_height=”0″][/vc_column][vc_column width=”2/3″][vc_single_image image=”57654″ img_size=”full” alignment=”center” media=”57654″ media_width_percent=”100″][/vc_column][vc_column width=”1/6″][/vc_column][/vc_row][vc_row row_type=”full-width-content” unlock_row=”” row_height_percent=”0″ override_padding=”yes” h_padding=”2″ top_padding=”0″ bottom_padding=”0″ overlay_alpha=”100″ gutter_size=”3″ shift_y=”0″ row_height_use_pixel=””][vc_column column_width_use_pixel=”yes” overlay_alpha=”50″ gutter_size=”3″ medium_width=”0″ shift_x=”0″ shift_y=”0″ zoom_width=”0″ zoom_height=”0″][vc_column_text]
CD is part of a four-step process used to discover and validate a pain point (or benefit trying to be achieved) that a particular customer segment is experiencing (or benefit they would like to experience). Ideally, a founder is aiming to find a pain point that ticks all these boxes:

  • The pain point should be one of the customer’s top 3
  • It should be experienced on a daily basis
  • The customer has tried to solve the problem themselves
  • The customer has tried or, are using a solution that doesn’t fulfil their need
  • are actively looking for an alternative solution

Once a founder has spoken to enough potential customers to verify this pain point with significant quantitative and qualitative evidence, founders are free to develop a prototype or Minimal Viable Product (MVP) with a minimal feature set they think will fix the problem their potential customers highlighted in the CD interviews and get it in front of customers.

By providing a prototype/MVP to your customers via a chosen marketing channel a founder will go through a process of building, learning, and measuring their efforts based on the feedback they get from their customers. The ultimate goal is to test the founder has:

  • Targeted the correct customer segment
  • The right value proposition for the right customer segment
  • Chosen the right channel to market to reach your customer
  • Achieving product-market-fit
  • Chosen the correct revenue stream and cost structure

Ultimately, the founder is aiming to achieve Product-Market-Fit (PMF). It’s important to have a logical, structured process do this as quickly and efficiently as possible as most startups don’t have infinite resources and they do have competition vying for the same market share.

CD is both art and science. There is a body of knowledge, particular types of questions, and a process that can be followed to achieve great outcomes. However, depending on many variables, the way in which founders approach CD will be different for each startup. I will share my process and documents as a guide, hopefully, to make it easier for others. At the end of the day, CD allows founders to question and articulate their assumptions around an idea/problem, formulate a test, carry out the test, gain insight based on evidence so founders can make informed, evidence-based decisions. It also allows for a more structured, objective, process orientated approach to building businesses, removing the over enthusiastic “my app is going to change the world” approach to building businesses. Finally, it replaced certainty and hubris with curiosity and humility, allowing the market to dictate what the entrepreneur should build and not the other way around.

Why Customer Development 

As mentioned above, many founders are trying to either applying something new to an existing market, something that already exists to a different market, or something new to a new market. As such, their potential customers have never seen what they have they potentially have to offer. Therefore, most people, represented by the early adopters, early majority, late majority, and laggards, won’t be interested as they are reasonably ok with the way things are. That’s why it is critical for founders to be looking for a very specific type of customer segment; innovators or evangelists. Founders should be trying to tick as many of the boxes mentioned in the dots points above for innovators or evangelists. These types of people are highly motivated to find an alternative solution and will, most likely, put up with a founder and a prototype/MVP. At the least, they’ll be willing to try to see if it fixes their problem. But, that begs the question; where are these early adopters and how do you engage with them?

[/vc_column_text][/vc_column][/vc_row][vc_row row_type=”full-width-content”][vc_column width=”1/6″][/vc_column][vc_column width=”2/3″][vc_single_image image=”57655″ img_size=”full” alignment=”center” media=”57655″ media_width_percent=”100″][/vc_column][vc_column width=”1/6″][/vc_column][/vc_row][vc_row row_type=”full-width-content”][vc_column][vc_column_text]Before madly rushing out to speak to anyone that might be a customer Ideally, a founder should have a good understanding of the CD process. I’ve outlined a high level overview here and then explained, in detail, the exact process I use to conduct my CD.

A founder should aim to do 20 – 30 minute face to face CD interviews, but this is often very time consuming or not possible. The next best option is via phone or Skype. The third option is to conduct online surveys. While survey  information is valid, it should be used to support or disprove your qualitative data. There is a significant different between the quality and value of information gathered face to face versus via interview. Founders are attempting to find said pain points within a customer segment. At times people have physiological and psychological reactions to questions and experiences. This is what founders should be looking for. This is where founders need to focus and ask more questions. This is where a founder’s assumptions will be (in)validated, where they will gain significant insight about their customer segment, and where this information will differentiate them from their competitors.

To give you an example, I worked with a startup that aimed reduce the challenges around divorce. While sharing their CD experience they stated that minutes into conducting the interview process, the interviewee burst into tears because of the stress around their divorce. This visceral reaction was a clear and significant pain point they were experiencing. This is an extreme example and I’m not suggesting that founder’s need to have every interviewee cry, it simply aims to highlight the type of experience and insight you can’t get by way of a survey.

The Customer Development process 

Getting out of the building the smart way 

If you go to a Startup Weekend or reading anything to do with the Lean Startup methodology or Business Model Canvas you’ll hear “there are no answers inside the building” and “get out of the building” touted at founders time and time again. If you go to a Startup Weekend, this is usually what happens; teams are formed, hypothesis is written about customer segments, and founders “get out of the building” to conduct customer development interviews on anything and anyone who happens to look vaguely like their customer segment, often to the shock of the innocent bystander.

I take a slightly different approach and will elaborate by way of using an example. The first step is to formulate a good hypothesis. Here are the formulas:

I believe [target market]will [do this action / use this solution]for [this reason].


I believe this to be true. If I do X I’m expecting Y result

One hypothesis I tested was “I believe founders that are trying to raise or have raised capital find it difficult to raise capital because it’s a time consuming and costly process.” With that, the next question to be asked and answered is “where do I find these types of people?”

Instead of taking the shotgun approach, I took a step back and looked at the entire lifecycle of my customer segment. Generally speaking, founders build their startup and head to a coworking space, incubator, or accelerator program. From there, they realise they need a significant capital injection to continue growth. At this point in their startup journey, they begin to experience the pains in and around capital raising. But, stepping into their shoes and thinking logically, practically, empathetically, this customer segment doesn’t know what it doesn’t know. It has yet to go through the 4 – 18-month process of pain and suffering, knocking on investor’s doors, pitching to the wrong investors, preparing for due diligence and the like.

A much more efficient way, I found, to gather exactly the information I needed was to go further along the lifecycle of a founder and speak to “CEOs of companies in Australia that have already raised capital.” This customer segment was able to tell me their war stories, provide beneficial insight, and confirm several assumptions while invalidating others. I gained an incredible amount of highly valuable insight from only 40 interviews. With this information, it was crystal clear as to the features that needed to be included in the MVP. When I explained this process to a founder he rephrased it as reverse engineering my customer segment’s life cycle which I think is a great way to think about it.

Continuing down the reverse engineering thought process, I asked myself “where would this customer segment go before and after they raised capital?” Usually, founders visit their lawyer, accountant, incubators/accelerators or start knocking on random investors doors. Digging deeper, I found that a lot of venture capital firms have a portfolio page on their website of the companies they have invested in. Where do founders go after they have received funding? The media. I started looking at two authoritative blog posts, Startup Daily and the Startup Smart. There is no end to the blog posts about founders that have raised capital.

Contacting, scheduling, and conducting Customer Development interviews 

Once you’ve found where your customer segments are, how do you get in contact with them and convince them to do a 30-minute interview? Part of the reason I’m writing this blog is that I’ve heard every type of BS excuse as to why founders don’t do CD. Often, I hear that founders are scared, they don’t know what questions to ask, they don’t know how to find their customer segment, etc.

I’ll share my experience interviewing “40 founders that have raised capital in Australia.” Capital raising can be difficult, really difficult. The process is extremely time-consuming, new to most founders, expensive, and is done while a founder is trying to run their a rapidly growing business. After a successful funding round, things get even busier as founders put their new found capital to use scaling up their business. Given this environment, out of all the CEOs, I contacted, only one declined an interview. Of all the people I interviewed, all were more than happy to help and spend 30 minutes on the phone. Some spent close to an hour providing insight I would not have otherwise have had access to. If I can do this with this type of extremely business customer segment, there is simply no excuse for anyone not to send an email or pick up the phone. PERIOD!

This is the process I took to get in contact and schedule interviews with my customer segment:

  • I hired a virtual assistant for 20 hours a week (4 hours a day, 5 days a week)
  • I added the VA to my Slack team, Trello board, and share a Google Sheet
  • I set up a task in my Trello board and assigned it to my VA
  • This is word for word, the list of instructions I provided. We did have a brief conversation over Slack to make sure that we were both on the same page:

When the founder responded, I would take over the process and arrange a time/date for the interview. Below is a copy of the email sent to the founders:



I saw a recent article about you in the [INSERT BLOG] and thought I’d reach out. I co-founded the business accelerator at Bond University, worked as the program manager at the University of Queensland’s ilab accelerator and noticed capital raising is the most difficult thing a founder faces. In trying to solve this problem I’m reaching out to founders that have or are trying to raise capital to hear about their experiences. From that, I’d like to build a solution to solve some of the challenges they faced. I was wondering if you had time for a phone some time this week?



NB – Notice I didn’t mention anything about “my new app that’s going to change the world.” I’m operating in the Customer Discovery and Customer Validation section of the process. If you mention your idea, the interview is done and the information therein useless because everything from that point forward is biased towards or away from that idea. Plus, nobody wants to hurt a founder’s feelings.

The interview process 

There is a right way to interview customers and then there’s a very wrong way. There is a structured approach of how to start and set expectations, what types of questions to ask, what types of questions not to ask, and specific outcomes founders are trying to achieve. I will highlight the whole interview process step by step. This assumes founders have contacted them, scheduled a face to face or phone call so there is some rapport.

Before I start the interview, I set myself up with my Customer Development Interview document. I create a heading with the person’s name, copy in their LinkedIn profile, and have a list of template questions just above the heading to guide me through the process. Before the interview, I spend about 15 – 20 minutes reading the person’s LinkedIn profile and the blog post they were mentioned in so I have topics of conversation specific to the interview.

Interview – introduction

Tres: I Founder, my name’s Tres, I’m calling from Merakium. How are you today?

Founder: Hi Tres, I’m well. Yourself?

Tres: Really well, thanks. How was your weekend/are you looking forward to the weekend?

Founder: Yes/no/maybeTres

Tres: Fantastic. Well, the reason, I’m calling is because we arranged a quick interview about your capital raising experience. Should we get started?

Founder: Yeah, sure.Tres: Great. So, I’m just going to go over the process really quickly. Let me know if you have any questions. I’m mindful of your time, so this will not go longer than 30 minutes. I’ve got you on speaker phone and you might notice me typing away furiously. These notes are only ever going to be used by me, I’m not going to show it to anyone, and I’m not going to mention your name or experience. I’d really just like to hear about your real experience good, bad, or ugly so I can get a good idea of the pain points and, perhaps, built a solution to address the problem.

Founder: Sure, no problem.

Interview – questions 

  • Ok, great. Can you give me a brief overview of your background
  • Can you give me a brief overview of your company (actually say the company’s name)
  • Can you tell me how you got started, why you started the company

It’s at this point I’m mapping what they’re saying with what I’ve read about them. I’m making a mental map of the major milestones of their experience so I can refer to them. Once I have a list of these milestones I’ll stop the interviewee due to time and my aim will be to dive deep into several scenarios. The aim here is to operate from the mindset of exploration, curiosity, and learning. I put all my knowledge and assumptions aside and pretend I know nothing of the topic of capital raising. I ask broad, open ended questions that allow the interviewee to provide a high-level overview of their experience. I’ll start at milestone one and explore that by asking the “5 whys” to get to the root cause. Wikipedia has a summary here. These “ 5 why” questions can be asked in many different ways:

  • Can you tell me more about that?
  • Can you explain that in more detail?
  • How come that happened?
  • How did that come about?
  • Can you elaborate on that point more?
  • Can you expand on that?
  • Can you go into a little more detail?
  • How does that work?
  • Why did that happen?

As I build a detailed picture of the first scenario, I’m looking for keywords associated with emotion and I’m listening for any emotional responses from the interviewee’s voice. I’m looking and listening for anything that will allow me to dig deeper and find out more about the pain points that the founder experienced. Once I’ve identified them while building out the high-level overview of scenario one, I’ll go back and dig deeper by focusing on the emotional sections of the interview. The aim of this part of the interview is to determine if this pain point is significant enough that people will pay money for you to fix it. This is what you’re looking for:

  • The pain point should be one of the customer’s top 3
  • It should be experienced on a daily basis
  • The customer has tried to solve the problem themselves
  • The customer has tried or is using a solution that doesn’t fulfil their need
  • are actively looking for an alternative

These are the types of questions I’d ask in this part of the interview: 

  • Did you experience that all the time?
  • How often did you experience it?
  • When did you experience that pain/emotion?
  • On a scale or 1 to 10, 1 being no pain and 10 being a “shark bit,” where would you rate this particular pain?
  • Was it in your top three pain points while raising capital? If so, where did it sit;1, 2, 3
  • What solution are you currently using, or what solutions currently exist in the market?
  • Why don’t they work for you?

I work through as many milestones as possible as time would allow. I leave about 5 minutes at the end of the interview to be able to close it out correctly as there are some key things that need to be done. Assuming I’ve gone through 3 milestones and I have a good idea of the pain points I’d ask one final question.

“If I had a magic wand and could build a technical solution that could fix the problems you experienced raising capital, what would that solution look like?” 

I asked most of the founders this question and it was remarkable how many of them said almost exactly the same thing. This level of insight allowed me to focus on EXACTLY what feature set to build next instead of assuming I knew what the market wanted.

Closing out the interview 

At this point, you want to validate your idea even further by asking one or a combination of questions. In this situation, interviewees value three things; their reputation, their time, and their money. Asking for one or a combination of any of these will give you some indication of whether or not you have clearly identified a significant enough pain point where the interviewee would actually be willing to risk their reputation by introducing you to someone else, give you more of their time to help you further, or put their money where their mouth is a sign up as a beta tester.

When validating using their reputation ask “who in your [legal/lawn bowling/knitting/engineering] network do you who is having that same problem that you might be able to introduce me to?”

When validating using their time say “I really appreciate you taking the time today. I got some really great insight into the process. Do you mind if I contact you in the future if I have any more questions?”

When validating using money ask “would you be willing to pay/sign up as an early adopter for a significantly reduced rate?” If they say yes, actually ask them for the money. This is not a hypothetical because what people say and what they do are often two different things. Get them to put their money where their mouth is.

In all scenarios, the interviewee is going to be on a sliding scale of “get the hell out of my office before I throw you out” to “hell yeah, when will it be ready so I can try it and tell all my friends about it.” Based on their response will determine whether or not you’ve (in)validated your assumptions.

One thing to note in the scenario above, I was interviewing “CEOs that have raised capital in Australia.” They had already gone through the capital raising process so did not need any product or service I might want to build therefore the validation questions at the end of the interview are somewhat irrelevant. Perhaps, asking for an introduction to another founder that might be trying to raise capital would be beneficial. Always close the interview by saying thank you to the interviewee for giving up their time to help you out.

MASSIVE caveat 

All a founder’s efforts to this point are not to suggest that they have a product that people will purchase. This particular CD process was testing ONE hypothesis of ONE customer segment to determine if ONE very small piece of a much larger puzzle, the founder’s world changing app, might actually work. This process is to give a founder “some” idea, a better idea, a direction based on evidence so they can make an informed decision about where to go to next as opposed to relying on their best guess or assumptions.

The biggest mistakes I see founders make conducting Customer Development

These are some of the most common mistakes I hear and see founders make when conducting CD interviews

  • Founders mention their idea/product/service. If an interviewee asks if you have something say “I have a couple of ideas but if I mention them now it will most likely skew the data in this interview.”
  • Founders pitch their idea.
  • Founders ask leading question e.g. you think that’s a great idea, don’t you? Or, that was a pain point, wasn’t it?
  • Don’t ask hypothetical or further based questions. In short, people say one thing and do another. If you ask them “if I built this, would you buy it?” everyone is going to say yes to appear polite, not hurt a founder’s feelings, and seem like a good person.
  • Founders don’t dig deep enough into each milestone they identify
  • Founders don’t take a structured approach to the whole process including formulating a hypothesis so they can measure what they learn, gain insight, make an informed decision, get it to market, and learn
  • Founders don’t do enough interviews. It will depend on the product/service founders want to bring to market. If founders have nailed the customer segment and conduct interviews in a consistent way they will follow what’s called the curve of diminishing returns whereby all the answers to all the questions will start sounding the same. At that point, they will have about 80% of the data being consistent and are able to gain significant insight and make informed decisions. The rest of the info, while relevant, will be outlying data.
  • Founders don’t do any interviews and assume they know best. While this might work out in some instances, ultimately, the market will dictate the final outcome.


As I mentioned, many founders start in much the same place; they have industry experience, see a problem, come up with an idea, and startup building the next best mouse trap because they think they know what the customers want, when in fact, it’s very often not the case.

I’ve provided my own process of exactly how I conduct my CD interviews, with some of the smartest and busiest people around, and demonstrated the insight I obtained in the blog “What I learned Interviewing 40 Founders That Raised Capital” so there should be no excuse as to why a founder doesn’t do CD interviews.

Finally, I provided some common mistakes I see over and over again that will, typically, render the information you collect in any interview null and void. With a bit of practice and some fine tuning of the process, I’m hoping this blog will assist founders in avoiding some of the major pitfalls when it comes to CD.[/vc_column_text][/vc_column][/vc_row]