Research Article
WHAT IS LEAN STARTUP: A SOLUTION TO STARTUP FAILURES OR NOT?
John M York
Corresponding Author: John M York, Rady School of Management, Institute for the Global Entrepreneur, Jacobs School of Engineering, University of California, San Diego, 9500 Gilman Dr, La Jolla, CA 92093, USA
Received: 17 September 2019; Revised: 05 April 2020; Accepted: 03 October 2019
Share :
  • 512

    Views & Citations
  • 10

    Likes & Shares

Lean Startup (LS) is a popular framework for efficiently developing entrepreneurial ideas. It involves a problem-solving approach using a scientific methodology for developing businesses, products and even business challenge solutions. It has garnered following in the startup community, along with several major corporations (e.g., General Electric) and within the United States government. This paper defines the LS methodology and its theoretical foundation. It examines several essential activities around customer discovery, minimum viable product (MVP) business model experimentation, validated learning and innovation accounting. LS involves two phases (search and execution) involved with LS and ties in several canvases (business model canvas, lean canvas and value proposition canvas) support LS practices.

LS fit some businesses well (e.g., web-based, tech, software and mobile spaces). Materials-based businesses and those involving long development and lead times, investment, intellectual property and regulatory constraints (e.g., biotech and pharmaceuticals) may not be as ideal. LS does offer potential application to areas within the travel, hospitality, hotel and restaurant business sectors. It offers a problem-solving approach that could be a strategy for organizations to approach various challenges.

LS possess several limitations involving several of its core elements and their use: customer discovery, experimentation, MVP and iteration/pivoting. Another relates to outcomes as much of the LS literature is anecdotal. While some empiric studies exist, the LS area would benefit from further research with structured studies to (1) define whether the methodology contributes to meaningful business outcomes and (2) its role and that of other influencing factors on startup success.


Keywords: Business Model Canvas, Customer Discovery, Entrepreneurial Experimentation, Entrepreneurship, Lean Startup (Start-Up), Lean Canvas, Lean Fit, Lean Limitations, Minimum Viable Product, Product/Market Fit, Startup Performance Outcomes, Value Proposition Canvas.

 

INTRODUCTION

 

            In the United States each year, entrepreneurs venture forth and start over six hundred thousand new ventures (Balle, 2015). Unfortunately, half are still in business within five years and one-third remains within ten years (Nazar, 2013; SBAUA FAQ, 2012). Of these new businesses, investors engage in less than 1% (Rao, 2013) and, of these firms, 75% will not survive (CB Insights, 2015; Deborah, 2012). When considering these statistics, it is incredible to think that one would want to start a new venture, no less in the hospitality and travel industry space.

               One of the problems with startups is how an entrepreneur approaches the business. Starting a new venture involves a tremendous amount of uncertainty that the startups needs to address. In particular, they fail to understand their marketplaces, competition and customers, as many are product-focused rather than market-focused. Interestingly, CB Insights (2017, 2018) identified, as part of a post-mortem of 101 startups, that the lack of market need (seen in 42% of the cases) was the primary reason for failure.

               Ries (The Lean Startup: How Today´s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses) saw this problem in the first two startups in which he worked (Rousch, 2011). He noticed that these companies, similar to many other startups, failed to understand that in many ways, starting a new business is similar to solving a problem. As a student of Steve Blank (The Four Steps to the Epiphany and The Startup Owner’s Manual) at the University of California, Berkeley, Ries drew upon Blank’s concepts of customer discovery in his next startup IMVU (Ries 2011). Based on his experience using this concept along with lean principals embodied in the Toyota Production Principal, he started a blog that turned into a national best-selling book. His and Blank’s efforts led to a tremendous following that embraced the concept of “The Lean Startup” (LS). Not only have thousands of entrepreneurs used this methodology, but also the National Science Foundation (NSF) Innovation Corps™ (I-Corps™) program, the United States Military (“Hacking for Defense”, H4D), numerous universities and multiple corporations (e.g., Dropbox, General Electric (GE), Intuit and Proctor and Gamble) have embraced this approach (Blank & Dorf, 2013; H4D, 2019; Lashinsky, 2018; National I-Corps™ grants, 2015; Nnakwe et al., 2018; VentureWell, 2015).

               In order to help entrepreneurs in the hospitality and travel space, this paper seeks to provide a review of the LS. It will examine its theoretical basis, essential components and issues to consider.

 

DEFINING LS AND ITS FOUNDATIONS

 

Over this past decade, LS became a popular framework for efficiently developing entrepreneurial ideas. It involves a problem-solving approach using a scientific methodology for developing businesses, products and even solutions to business challenges. The approach focuses on shortening the product development cycle visa vie business-hypothesis-driven experimentation, iterative product releases, validated learning and customer feedback (Everything explained, 2019; Investopedia, 2015). LS draw on insights from the Toyota Production System’s lean manufacturing principles and agile software development processes (Krafcik, 2015; Ohno, 1998; Investopedia, 2015). 

Several academic theories support LS. These include creation, discovery, dynamic capabilities, effectuation, bricolage, business model and customer development (Alverez & Barney, 2007; Baker & Nelson, 2005; Blank, 2005; Eisenman, 2012; Frederikson & Brem, 2017; Ghezzi, 2015, 2018; Ladd, 2016; Rappa, 2001; Saravanthy, 2001; Shane & Venkataraman, 2000; Teece, 1997; Yang et al., 2018). Frederikson and Brem (2017) examined the underlying theoretical basis and identified evidence specific to the five essential pieces and graded data with a subjective rating based on their evaluation of the quality and quantity of supportive literature (Frederikson & Brem, 2017). These include: (1) user and customer involvement (very strong); (2) effectuation (strong); (3) iteration in new product development (strong); (4) early prototyping for proof-of-business (MVP); and (5) experimentation in new product development (Frederikson & Brem, 2017).

 

ESSENTIAL PIECES AND PHASES

 

Essential to LS are two phases (Figure 1) involving (1) search and (2) execution. LS use several core pieces within these phases: (1) customer discovery; (2) experimentation; (3) a minimum viable product (MVP); (4) validated learning; and (5) innovative accounting. 

In the first phase, the startup focuses on searching for customer needs, product/market fit and a repeatable sales model. The team starts with the first essential part of LS, that of customer development, which concentrates on understanding customer problems and needs- pains, gains and the job to do. Blank introduced this concept, which is as important as product development (Blank, 2005; Blank, 2013). He emphasizes that discovery should start early in the process. It involves the creation, testing and refinement of hypotheses or guesses through direct conversations with customers by “getting out of the office or building” or “GOOB” to get inside the customers head (Blank, 2013). With such data, the startup team can build an MVP to validate the problem and identify viable solutions, including as product, value proposition and business models. This process should connect the customer needs with the product. In particular, it is to define a value proposition, in which Figure 2 and Table 1 from Bain consulting highlight as elements of value in the business-to-consumer space. 

Integral to LS is the process of experimentation. Once the startup team has completed discovery, it can then focus customer validation based on an MVP and building a replicable sales model visa vie experimentation. Heavy use of effectuation-logic is evident, with a clear and explicit emphasis on experimentation. Ries characterizes this part as the “build-measure-learn” loop in which the entrepreneur sets up a hypothesis (or guess), an experiment (e.g., A/B, landing page, Kickstarter campaign) to test it and a threshold metric for success or failure. The idea of this loop is that the startup begins with an MVP and gets it into the hands of customers quickly for feedback that will help to either reject or validate assumptions and to gauge traction. The purpose of this cycle is to minimize the time through the feedback loop-build, measure and learn faster. Essential is the MVP, a most minimal product or solution (e.g., cupcake as a sample for a wedding cake) to address the customer need.

In testing hypotheses, innovation accounting and metrics are essential. LS promote the metric-based evaluation technique to help validate learning. Startups can test their hypotheses in a quantitative way, such as in evaluating click-through rates, sign-ups and customer acquisition costs via a minimal landing page. Ries cautions against “vanity metrics” (Ries, 2011; Mueller & Thoring, 2013). He points to the use of “innovation accounting” to measure the progress while validating learning and defines actionable metrics linked to a specific business model. In testing hypothesis, Ries differentiates among three “engines of growth” (viral, sticky and paid) and offers metrics for each of them. He also highlights the value of A/B testing, something that frequently appears in the evaluation of software programs (Mueller & Thoring, 2013).

According to Croll and Yoskovitz (2013) and Rompho (2018), these metrics can vary based on the type of startup (e.g., e-commerce, software-as-a-service, media site, user-generated content, mobile app, two-sided marketplaces) and the stage of its development (e.g., empathy, stickiness, virality, revenue, scalable). It is in this experimentation phase in which the entrepreneur validate one’s learnings. Customer interviews and hypothesis testing to drive learning to provide qualitative data and quantitative data that the startup can make an informed decision. 

Based on what the entrepreneur learns, one can iterate, pivot, or continue forth with the idea since the experiment validated the hypothesis. It is essential to recognize that the iteration and pivoting are not the same actions. Iteration involves small changes in the product or business model based on learning from interviews and experiments conducted. Pivot involves moving off one’s initial premises and MVPs to alternative ideas. This action involves significant and structured changes from the initial hypothesis and MVP to new ones concerning product and business model.

Throughout this process, the goal is about learning from interviews, research and testing of ideas. This effort aids the startup team to efficiently make “go forward” or “fail fast” decisions. Ries points out that central to the learning process is his “Build-Measure-Learn” feedback loop (Figure 3), similar to what occurs in AGILE product development (Ries, 2011). If the idea is to fail, then the startup should “fail fast” to minimize resources and time wasted and to maximize learning. 

By talking to customers and testing, the startup can identify where its product and business model have achieved product/market fit (P/MF) or traction. Hence, the ultimate end of this learning process is P/MF. Netscape founder and venture capitalist, Marc Andreessen, describes P/MF as “being in a good market with a product that can satisfy that space” or that “the startup has built something people want” (Andreessen, 2015). Blank refines this definition as to whether the startup identified a repeatable and scalable sales model before the venture can proceed to the next phase and scale up the business (Blank & Dorf, 2012).

The second phase involves execution. This part involves customer creation and company scaling. The startup’s focus changes from learning to scaling. The entrepreneur concentrates on creating customers, driving demand and building the company. If the lean process has been successful, then scaling should occur more efficiently and effectively. Nonetheless, the startup will continue to talk to customers, test hypotheses and run experiments to refine the product and business model. Sean Ellis another leading entrepreneur (Dropbox) and author, characterizes this effort as hacking for growth (Ellis & Brown, 2017).

 

THE ROLE OF CANVASES

 

Several canvases, or one-page frameworks for recording hypotheses and changes, provide the back-end to support LS activities as the front end. Three are relevant to LS: (1) the value proposition canvas (VPC, Figure 4); (2) the business model canvas (BMC, Figure 5); and (3) the lean canvas (Figure 6).

Osterwalder and Pigneur (2010, 2014) developed both the VPC (Value Proposition Design) and the BMC (Business Model Generation) and have taught entrepreneurs all over the world on their use. The VPC involves two components, customer on the right and product/service on the left. With this canvas, the startup can track for each of its essential customer segments the critical pains, gains and “jobs-to-do” on the right side (the circle). The entrepreneur uses this tool during customer discovery to focus on the critical customer issues to develop out and then test the hypotheses developed in this section. Once one has gained some insights, the entrepreneur then can sketch out on the left side (the box) the pain relievers, gain creators and essential elements of the product or service on the left side. In essence, one can map the customer information and needs he/she has gathered through interviews. These components help to share the most minimal and critical attributes that the entrepreneur can more clearly build an MVP.

The BMC consists of nine pieces that define the business model and would support the value proposition. The right-hand segment, the “value side,” focuses on value creation and extraction. It includes: (1) customer segments; (2) value proposition; (3) customer relationships; (4) channels (distribution); and (5) revenue streams (models). This part focuses on the pieces needed to create and extract value. Essential is to connect the customer segments with the value proposition, which should closely align with what the entrepreneur does with the VPC.

Further, the entrepreneur needs to consider how to create customer relationships to acquire (and keep) them. This segment involves utilizing tactics and tying them into the marketing funnel of “get,” “keep” and “grow.” The “get” piece is essential as it outlines the customer journey from awareness, to interest, to decision, to finally acquisition. It is in this part that the entrepreneur needs to tie in various marketing tactics to guide the customer along. The channels (as in distribution, not communication channels) piece is critical as it defines how the entrepreneur is going to get the product to the customer, either directly or via intermediaries (e.g., wholesalers, retailers) and whether it involves a physical or digital route. Finally, the revenue model accounts for how the entrepreneur will capture value, which can be via direct purchase, subscriptions, two-sided or multiple other options. Overall, it supports the value side of the canvas and, most importantly, the value and revenue model must be sustainable enough to cover a firm’s expenses.

The left-hand segment, the “efficiency” or “operational” side focuses on the operational side of the venture to deliver on the value proposition. It includes: (1) key resources; (2) key activities; (3) key partners; and (4) cost structure. Key resources involve people, physical (e.g., plant, equipment), intellectual property and capital. Key activities can vary depending on the type of firm and whether it can either perform them directly or outsource to partners. Such activities include manufacturing, marketing/sales, consulting, customer service, accounting and legal, among others. Essential is that the entrepreneurs measure them to assess performance. Key partners include one’s supply chain, but also strategic alliances, joint ventures and coopetition (e.g., trade organization). Finally, there is the cost structure, which considers the other pieces within the operational side. The cost structure usually accounts for the use of such resources and activities via fixed and variable costs.

The Lean Canvas is a take on the business model canvas Ash Maurya’s Maurya (Running Lean: Iterate from Plan A to A Plan That Works) (2012). He outlined this one-page template to help entrepreneurs deconstruct their ideas into its essential assumptions that one would develop further into a business plan (Mullen, 2016). Maurya breaks the canvas into two sides, product (left side) and market (right side). The product side involves the following parts: (1) customer pain/problem (existing alternatives); (2) solution including technical feasibility; (3) key metrics; and (4) cost structure. The market side involves: (1) unique value proposition; (2) unfair advantage; (3) channels or the ease of reach (path to customers); (4) revenue streams; and (5) customer segments including market size (early adopter). As one can see, the lean canvas utilizes parts of the BMC but takes on new pieces such as the problem, the solution, key metrics and unfair advantage. By doing so, it allows for consideration of the product and differentiation from the external market. Furthermore, the channels piece considers both elements of product distribution as well as customer acquisition. Maurya weighs the respective parts from highest to lowest as: (1) customer pain/problem; (2) ease of reach (channels); (3) price/gross margin (revenue streams/cost structure; (4) market size (customer segments; and (5) technical feasibility.

 

BUSINESS FIT

 

One question that exists is whether all firms can use the LS methodology. Considering its roots, LS might be limited to software-driven ventures (e.g., Could-Fire, Dropbox and IMVU) that address a business-to-consumer market (Ries, 2011). Croll and Yoskovich (2013) highlight six digital models (e.g., e-commerce, the two-sided marketplace, software as a service, free mobile app, media, user-generated content) that use LS practices, particularly innovation accounting. In fact, scholars have pointed out that specific practices such as experimentation, use of an MVP and iterating/pivoting) appear most applicable to software development (Frederickson & Brem, 2017).

Interestingly, several corporations use LS in areas beyond its software roots. Ries highlights several notable firms (startups and established) in his book (Ries 2011). Examples include General Electric (GE), Hewlett Packard, Intuit, Paypal, Proctor & Gamble, Telefonica, Toyota and Zappos (Frederikson & Brem, 2017; Lashinsky, 2018; Ries, 2011).

The GE FastWorks offers an excellent example (2011). As a result, GE experienced tremendous success in its gas turbine and appliance divisions (Lashinsky, 2018; Power 2014). The gas turbine division saw its product development cycle run two-years faster and 40% less expensive, along the division seeing $2 billion in revenues. The appliance division realized product development at half the cost and twice the rate, while it doubled its sales growth rate.

However, for some businesses, such as the material technologies space (e.g. chemical, materials, semiconductor, silicon chips), LS is not ideal. Harms et al. (2015) argue that materials and science-based ventures do not fit well. This rationale is because such firms: (1) operate under a high degree of technological uncertainty to resolve; (2) often serve business markets; (3) closely link product and process innovation, which make for challenges for an MVP and lead to intellectual property issues (e.g., patents to address) (Harms et al., 2015).

An example of such a firm exists in the life sciences space. Biotech and pharmaceutical continually have to manage technological uncertainties. These firms require a long time to market (approximately ten years) and significant investment ($2.5 billion) (Mehta, 2011; Vedd et al., 2019). They also need to reconcile with regulators and other value chain partners who can influence the commercialization, development and profitabilities.

As to the travel, hospitality, hotel and restaurant business sectors, LS may have areas that do apply and not. Examples of business areas where LS may apply well include the online, mobile and software spaces. It is in these areas that a firm can roll out and test an MVP or its business model. For example, Airbnb did such with some of its growth marketing practices in using pictures with A/B tests and saw dramatic results (Croll & Yoskovitz, 2013). Hospitality or touring services might be another space where LS might work well. Even food startups might benefit from LS practices. One published example involved a case study involving the use of LS and the BMC in the validation of the feasibility of a tour bus company in Indonesia (Dewobroto & Siagian, 2015). Another example involves The Brown Butter Cookie company of Cayucos, California, which used an MVP to test and gain traction (via product demand and sales) with their specialty cookies at the Cass House (Personal experience).

On the other hand, LS may or may not fit with setting up a large hotel or restaurant. Significant investment, development time and regulatory considerations involved with launching such ventures might limit the use of LS. Business plans and cases may make better sense with these types of businesses.

However, firms can use LS can as a tool for problem-solving rather than for product development. Many successful startups are just finding and solving of problems that create new products and business. Hence, LS, as with lean, could be such a methodology to address organizational problems. In many ways, consulting firms, such as McKinsey, use the customer discovery processes and interviews to uncover problems to identify solutions and then to experiment with MVPs or minimum viable solutions (MVS) in test cases (Friga, 2009). The GE FastWorks approach might exemplify such an application since the conglomerate rolled it out throughout the corporation (Lashinsky, 2018; Power, 2014). Another example of an organization using LS methods is the United States Military (H4D, 2019). In recent years, it has rolled out a variation of LS in its “hacking for defense” program where it employs discovery, experimentation and use of a Mission Model Canvases (MMC). This canvas represents a variation of the BMC with changes in the value or right side of the canvas to reflect beneficiaries (instead of customers), buy-in (for customer relations and acquisition), deployment (rather than channels) and mission achievement (in place of revenue model). Many firms can also embrace such a model to address corporate problems and missions to accomplish.

 

LIMITATIONS

 

No discussion on LS would not be complete without touching on some of the limitations of the methodology or its use in practice. This paper has already identified business verticals to which the approach would fit. There are several other areas that consultants and scholars have identified in both the peer review and non-peer literature. These include several practices and the MVP. The practices include that of: (1) customer discovery; (2) experimentation; and (3) iteration. Furthermore, there is a need for stronger empirical outcomes data.

Concerning the MVP, several consultants and scholars have identified challenges in its use. The problem lies in defining what the MVP is, launching it too early, fear of launching an inferior product and launching it in markets with lots of competition or customers not used to be innovators or early adopters (Finernan, 2013; Ng, 2015; Rao, 2015). Furthermore, there are concerns of some technical challenges in software development that might devalue the product, lead to waste or limit innovation (Warberg and Thorup, 2015). Scholars have pointed out that it might limit the solution space (Frederikson & Brem 2017).

In examining several of the core practices, customer discovery is quite concerning. Consultants have highlighted issues of not conducting an interview properly (Ng, 2015). Scholars have highlighted multiple biases involved with the interview and the processing of the data (Chen et al., 2015; York & Danes, 2014). Furthermore, others have pointed out that customers might also have their own cognitive biases due to different expectations and frame-of-reference (Croll & Yoskovitz, 2013). Finally, there are issues with getting adequate customer samples for interviewing and perhaps not genuinely uncovering big ideas due to interviewing skills and conduct (Nirwan & Dhewanto (2014); Gustafsson & Qvillberg, 2014).

Experimentation is another that is of concern. Many entrepreneurs do not know what goes into the development of an experiment. Consultants have seen problems in the creation of experiments as related to hypotheses developed, design, sample size, statistics and entrepreneur bias (Ng, 2015; Schaffer, 2014). Others have noted that some environments are too complex and chaotic for entrepreneurs to form and test meaningful hypotheses and that coming up with perfect experiments provides a great excuse not to take action (Vlaskovits 2018). Others observed entrepreneurs experiencing challenges in creating and validating the problem and then the solution (Nirwan & Dhewanto, 2014). Finally, some have noticed that experiments only provide a “pinhole” effect due to a limited audience of very early adopters that may not be representative (Heitmann, 2014).

Iteration and pivoting do also have limits. Some have indicated concern about the lack of learning and change (Heitmann 2014). Several investigators have noticed that some entrepreneurs might have difficulty with pivoting due to lack of a significant problem (Gustafsson and Qvillberg, 2012; Nirwan & Dhewanto 2014). Another noticed that some entrepreneurs, despite their knowledge of LS, fail to pivot their business models (Lliac et al., 2012). Others highlight that LS might produce “false negatives,” translating the rejecting of good ideas without learning from the data because the methods did not provide clear rules for defining go/no go, success (P/MF), stopping testing and prematurely scaling (Ng, 2015; Ladd, 2015). Finally, there is getting the whole team on the same page related to learnings and pivots (Ng, 2015).

The final limitation concerning LS relates to outcome data. Much of the LS literature is anecdotal. Several books that tout LS (Blank, 2007; Croll & Yoskovitz, 2013; Maurya, 2012; Ries, 2011) tie in stories and examples to illustrate the use of LS and its success. Scholars and VentureWell have reported the Innovation CORPS™ experience, which used LS, with up to 600 startups and $210 million in follow on funding (Nnakwe et al., 2018; VentureWell, 2019). The Startup Genome project surveyed over 650 web startups and found that founders who learn and pivot a few times experience more successful business outcomes than those who do not (Marmer et al., 2011). However, some academics do regard these experiences, along with the other case reports or examples in the literature, as anecdotal evidence (Frederikson & Brem 2017).

There appear to be a limited number of empiric investigations that examine performance outcomes with LS or LS-like practices (e.g. adaptation). Many of the so-called empiric studies involve case studies or surveys, none of which published in a significant entrepreneurship journal. However, a few interesting empiric studies offered useful insights concerning performance outcomes, though each study possessed limitations (Andries & Debackere, 2007; Ladd, 2015; Ghezzi et al., 2015; Ladd, 2015; Nilsen & Ramm, 2015). Work before the emergence of LS showed that adaptation might significantly influence both startup and within-firm survival (Andriew & Debackere, 2007). Cleantech accelerator data revealed that testing and customer discovery might (or not) make a difference in a pitch competition performance, depending on how entrepreneurs use them (Ladd, 2015). LS in the mobile space may lead to shorter times for the development of products, acquisition of first customers and firm organization than a business plan (Ghezzi et al., 2015). These differences in this study were not significant due to small sample numbers (n=4). Finally, LS use does not necessarily correlate with success (Nilsen & Ramm, 2015).

 

CONCLUSION

 

LS has garnered quite a following in the startup community, along with several major corporations and United States government departments. This discussion defined the problem of business failures and posed the question of whether LS was a potential solution. Considering the failure rate of startups, even with those who receive funding, LS has provided a more structured framework to address market and customer uncertainties. This paper defined the methodology and its theoretical foundation. It examines several essential activities around customer discovery, MVP and business model experimentation and validated learning. It also discussed the two phases involved with LS and customer development: search and execution. Furthermore, it has tied in several essential canvases that serve as a backend to support LS practices and to define (and test) appropriate value proposition(s), MVP and business models.

LS appears to best suit businesses where the entrepreneur can experiment, iterate and pivot with ease. Applications-based business sections, such as the web-based, tech, software and mobile spaces, seem to be the so it is the ideal areas to use LS. Additionally, service businesses may be able to use LS effectively as well. Materials-based businesses and those involving long development and lead times, investment, intellectual property and regulatory constraints (e.g. biotech and pharmaceuticals) may not be as ideal. However, the effectiveness of GE with its FastWorks program represents a notable success example. LS do offer a potential application to areas within the travel, hospitality, hotel and restaurant business sectors. Those that fit within the web, tech and service space might find LS to be an effective method for identifying customers, their needs and the testing of MVPs and business models. In contrast, those that require more substantial investment and development may not find LS as the ideal approach. However, readers should note that LS offers a problem-solving approach that the United States Military is using and could be a strategy for organizations to approach various challenges.

LS does possess several limitations, of which some are inherent to the methodology and others that relate to its implementation. In addition to the business fit area, the most notable include issues with LS and the implementation of several of its core elements. Customer discovery has issues with the conduct and implementation of interviews, the gathering of adequate interview samples and interview (and interpretation) bias. Experimentation has issues with hypothesis development, design of experiments, adequate samples for statistical significant, interpretation of data. The MVP has multiple issues surrounding its definition, how entrepreneurs should use it in testing and where it may be appropriate to use or not. Iterating and pivoting have considerations concerning entrepreneurs understanding the difference between the two and implementing them because the team is not in agreement and the startup have not identified a real big issue with the customer.

The final area relates to outcomes. Much of the LS literature is anecdotal. A few empiric studies do exist but are limited in their design and findings. The LS area would benefit from further empiric research with structured studies to: (1) define whether LS contributes to meaningful business outcomes (e.g., financial independence, funding, growth, positive cash flow for 6 months, revenue, survival, time to first customer); and (2) the role of LS and other influencing factors on startup success. 

(2019). Hacking for defense. Retrieved from: http://hacking4defense.stanford.edu/details.html

Alvarez, S.A. & Barney, J.B. (2007). Discovery and creation, alternative theories of entrepreneurial action. Strategic Entrepreneurship Journal, 1(1-2), 11-26.

Andreessen, M. (2015). On product/market fit for startups. Retrieved from: https://www.linkedin.com/pulse/marc-andreessen-product-market-fit-startups-marc-andreessen

Andries, P. & Debackere, K. (2007). Adaptation and performance in new businesses: Understanding the moderating effects of independence and industry. Small Business Economics, 29(1/2), 81-99.

Baker, T. & Nelson, R.E. (2005). Creating something from nothing: Resource construction through entrepreneurial bricolage. Administrative Science Quarterly, 50(3), 329-366.

Balle, L. (2015) Information on small business startups. Houston Chronicle. Retrieved from: http://smallbusiness.chron.com/information-small-business-startups-2491.html

Blank, S. & Dorf, B. (2012) The Startup Owner’s Manual. Pescadero, CA: K&S Ranch, Inc.

Blank, S. (2005). The Four Steps to the Epiphany-Successful Strategies for Products that Win. Pescadero, CA: K&S Ranch, Inc.

Blank, S. (2013). Why the lean startup changes everything? Harvard Business Review. Retrieved from: https://hbr.org/2013/05/why-the-lean-start-up-changes-everything

CB Insights. (2015). The venture capital funnel: Your chances of raising follow-ons, exiting and becoming a unicorn. Retrieved from: https://www.cbinsights.com/blog/venture-capital-funnel

CB Insights. (2017). 242 startup failure post-mortems. Retrieved April 5, 2018, from CBInsight Research Brief: https://www.cbinsights.com/research/startup-failure-post-mortem/

CB Insights. (2018). The top 20 reasons startups fail. Retrieved from: https://www.cbinsights.com/research/startup-failure-reasons-top/

Chen, T., Simon, M., Kim, J. & Poploskie, B. (2015). Out of the building, into the fire: An analysis of cognitive biases during entrepreneurial interviews. New England Journal of Entrepreneurship, 18(1), 59-76.

Croll, A. & Yoskovitz, B. (2013). Lean analytics: Use data to build a better startup faster. Sebastopol, CA: O'Reilly Media.

Deborah, G. (2012). The venture capital secret: 3 Out of 4 startups fail. Retrieved from: The Wall Street Journal. Retrieved from: http://www.wsj.com/articles/SB10000872396390443720204578004980476429190#articleTabs_comments%3D%26articleTabs%3Darticle mortem/

Dewobroto, W.S. & Siagian J. (2015). Business model generation and lean startup method as the basis for business development feasibility study, case study of PO. Gajah Mungkur Sejahtera. Proceeding 8th International Seminar on Industrial Engineering and Management. Retrieved from: https://www.researchgate.net/profile/Wisnu_Dewobroto/publication/293649474_Business_Model_Generation_and_Lean_Startup_Method_as_the_basis_for_Business_Development_Feasibility_Study_Case_study_of_PO_Gajah_Mungkur_Sejahtera/links/56ba197108aed47e41f38cb8/Business-Model-Generation-and-Lean-Startup-Method-as-the-basis-for-Business-Development-Feasibility-Study-Case-study-of-PO-Gajah-Mungkur-Sejahtera.pdf

EBAN Secretariat. (2010). Statistics compendium 2010, http://www.eban.org/components/com_virtuemart/shop_image/product/EBAN_StatisticsCompendium_Full_060111.pdf, 20.04.2012

Eisenmann, T., Ries, E. & Dillard, S. (2011). Hypothesis-driven entrepreneurship: The lean startup. Harvard Business School Background Note, 812-095, 1-23.

Ellis, S. & Brown, M. (2017). Hacking growth: How today’s fastest-growing companies drive breakout success. New York. Crown Business.

Finneran, J. (2013). The fat startup: Learn the lessons of my failed lean startup. Retrieved from: http://wordsting.com/copywriter-blog/fat-startup-learn-the-lessons-of-my-failed-lean-startup

Frederiksen, D.L. & Brem, A. (2017) How do entrepreneurs think they create value? A scientific reflection of Eric Ries’ Lean Startup approach. Int Entrep Manag J, 13, 169.

Friga, P.N. (2008). The McKinsey engagement: A powerful toolkit for more efficient and effective team problem solving. McGraw Hill Professional.

Ghezzi, A. (2019). Digital startups and the adoption and implementation of lean startup approaches effectuation, bricolage and opportunity creation in practice. Technological Forecasting and Social Change, 146, 945-960.

Ghezzi, A., Cavallaro, A., Rangone, A. & Balocco, R. (2015). A comparative study on the impact of business model design & lean startup approach versus traditional business plan on mobile startups performance. Proceedings of the 17th International Conference on Enterprise Information Systems.

Gustafsson, A. & Qvillberg J. (2012). Implementing lean startup methodology - An evaluation. Master’s Thesis at the Chalmers University of Technology. Retrieved from: http://publications.lib.chalmers.se/records/fulltext/164603.pdf

Harms, R., Marinakis, Y. & Walsh, S.T. (2015). Lean startup for materials ventures and other science-based ventures: Under what conditions is it useful? Translational Materials Research, 2(3).

Heitmann, J. (2014). The lean startup. A pragmatic view on its flaws and pitfalls. MA Thesis. The University of Twente. Retrieved from http://essay.utwente.nl/66221/1/Heitmann_BA_MB.pdf

Investopedia. (2019). What is ‘lean startup’? Retrieved from: https://www.investopedia.com/terms/l/lean-startup.asp 

Krafcik, J.F. (1988). Triumph of the lean production system. Sloan Management Review, 30(1), 41.

Ladd, E. (2015). Routines of new venture conceptualization: Evidence and extension of an entrepreneurial dynamic capability. Electronic Thesis/Dissertation. Retrieved from: https://etd.ohiolink.edu/

Ladd, T. (2016). The limits of the lean startup method. Harvard Business Review. Retrieved from: https://hbr.org/2016/03/the-limits-of-the-lean-startup-method

Lalic, M., Calopa, M.K. & Horvat, J. (2012). Lean startup in Croatia - Does the Croatian startup scene develop effective enterprises? Tem Journal, 1(3), 200.

Lashinsky, A. (2018). 5 big companies practicing ‘the startup way.’ Fortune. Retrieved from: https://fortune.com/2018/02/22/startup-way-procter-gamble-general-electric/

Lean Startup. (2019) Everything explained. Retrieved from: https://everything.explained.today/Lean_startup/

Marmer M., Hermrmann B.L., Berman R. (2011) Startup genome report 01, a new framework for understanding why startups succeed. Retrieved from: http://www.wamda.com/web/uploads/resources/Startup_Genome_Report.pdf  

Maurya, A. (2012). Running lean: Iterate from plan A to a plan that works. Sebastopol, CA: O'Reilly.

Mehta, S.S. (2011). Commercializing successful biomedical technologies: Basic principles for the development of drugs, diagnostics and devices. Cambridge, England: Cambridge University Press.

Mullen, R. (2014). Cost to develop new pharmaceutical drug now exceeds $2.5B. Scientific American. Retrieved from: https://www.scientificamerican.com/article/cost-to-develop-new-pharmaceutical-drug-now-exceeds-2-5b/

Mullen, S. (2016). An introduction to lean canvas. Medium. Retrieved from: https://medium.com/@steve_mullen/an-introduction-to-lean-canvas-5c17c469d3e0

Müller, R.M. & Thoring, K. (2012). Design thinking vs. lean startup: A comparison of two user-driven innovation strategies. 2012 International Design Management Research Conference. August 8-9, 2012. Boston, MA. Leading Through Design, 151.

National I-Corps™ Grants. (2019). NSF I-Corps Sites Program (I-Corps Sites) Program Solicitation NSF 14-547. Retrieved from: http://www.nsf.gov/pubs/2014/nsf14547/nsf14547.htm

Nazar, J. (2013). 16 surprising statistics about small businesses. Forbes/Entrepreneurs. Retrieved from: http://www.forbes.com/sites/jasonnazar/2013/09/09/16-surprising-statistics-about-small-businesses/#1577827a3078

Ng, G. (2014). Five pitfalls of running lean startup experiments. Forbes. Retrieved from: http://www.forbes.com/sites/groupthink/2014/04/28/five-pitfalls-of-running-lean-startup-experiments/

Nilsen, J.T. & Ramm, N.A. (2015). The lean startup: A success factor? A quantitative study of how use of the lean startup framework affects the success of Norwegian high-tech startups. Master’s Thesis. The University of Oslo. Retrieved from: https://www.duo.uio.no/handle/10852/45111

Nirwan, M.D. & Dhewanto, W. (2015). Barriers in implementing the lean startup methodology in Indonesia - Case study of B2B startup. DataProcedia - Social and Behavioral Sciences, 169, 23-30.

Nnakwe, C.C., Cooch, N. & Huang-Saad, A. (2018). Investing in academic technology innovation and entrepreneurship: Moving beyond research funding through the NSF I-CORPS™ program. Technology and Innovation, 19, 773-786.

Ōno, T. (1988). Toyota Production System: Beyond Large-Scale Production. Cambridge, Mass: Productivity Press.

Osterwalder, A., Pigneur, Y. & Clark, T. (2010). Business Model Generation: A Handbook for Visionaries, Game Changers and Challengers. Hoboken, NJ: Wiley.

Osterwalder, A., Pigneur, Y., Bernarda, G., Smith, A. & Papdakos, T. (2015). Value Proposition Design. Hoboken, NJ: Wiley.

Power, B. (2014). How GE applies lean startup practices? Harvard Business Review. Retrieved from: https://hbr.org/2014/04/how-geapplies-lean-startup-practices

Racolta Paina, N. & Andries, A.M. (2017). Identifying entrepreneurship readiness for the application of the lean startup practices in the service industry – Case study Romania. Ecoforum Journal, 6(3). Retrieved from http://ecoforumjournal.ro/index.php/eco/article/view/688

Rao, A.S. (2014). Minimum viable product (MVP) for product startup: An Indian perspective. Presented in Forum for Knowledge Sharing IX Annual Conference. Conference Theme: Technology: Corporate and Social Dimensions, Venue-National Institute of Advanced Studies (NIAS), Bangalore. Retrieved from: http://fgks.in/images/pdf/conf/ASRao.pdf

Rao, D. (2013). Why 99.5% of entrepreneurs should stop wasting time seeking venture capital? Retrieved from https://www.forbes.com/sites/dileeprao/2013/07/22/why-99-95-of-entrepreneurs-should-stop-wasting-time-seeking-venture-capital/#4aa9bd7546eb

Ries, E. (2011). The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. New York, NY. Crown Business (Division of Random House, Inc.).

Rompho, N. (2018). Operational performance measures for startups. Measuring Business Excellence, 22(1), 31-41.

Sarasvathy, S.D. (2001). Causation and effectuation: Toward a theoretical shift from economic inevitability to entrepreneurial contingency. The Academy of Management Review, 26(2), 243-263.

Shafer, H. (2014). Innovation games: A lean startup secret weapon presentation. Innovation Games Summit. Retrieved from: https://www.youtube.com/watch?v=CeHFY8HA4tI

Shane, S. & Venkataraman, S. (2000). The promise of entrepreneurship as a field of research. Acad Manag Rev, 25(1), 217-226.

Small Business Association Office of Advocacy. (2019). Frequently asked questions. Retrieved from: https://www.sba.gov/sites/default/files/FAQ_Sept_2012.pdf

Small Business Association Office of Advocacy. (2019). Small business profiles US. Retrieved from: https://www.sba.gov/sites/default/files/advocacy/2018-Small-Business-Profiles-US.pdf

Teece, D.J. Pisano, G. & Shuen, A. (1997). Dynamic capabilities and strategic management. Strategic Management Journal, 18(7), 509-533.

The United States Small Business Administration. (2012). Business survival. Retrieved from: https://www.sba.gov/sites/default/files/Business-Survival.pdf

Vedd, R.R., Fountaine, H.D., Liu, D., & Wu, A. (2019). FDA drug approval and its relation to a pharmaceutical company’s stock price. Journal of Finance and Accountancy, 24, 1-42.

VentureWell. (2019). I-Corps™. Retrieved from: http://venturewell.org/I-Corps(TM)/

Vlaskovits, P. (2018). Lean startup: Pivot, persevere or portfolio? Retrieved from: http://vlaskovits.com/lean-startup-pivot-persevere-or-portfolio/

Warberg, N. & Thorup, N. (2015). Preventing challenges in lean startup methodology. Copenhagen Business School. Master’s Thesis. Retrieved from: https://studenttheses.cbs.dk/bitstream/handle/10417/5765/Nicklas_Warberg_Nicolai_Thorup.pdf?sequence=1

Yang, X., Sun, S.L. & Zhao, X. (2018). Search and execution: Examining the entrepreneurial cognitions behind the lean startup model. Small Business Economics. Retrieved from: https://doi.org/10.1007/s11187-017-9978-z

York J.L. & Danes J.E. (2014). Customer development, innovation and decision-making biases in the lean startup. Journal of Small Business Strategy, 24(2), 21-39.