Decision-Making Under Uncertainty: Balancing Data, Intuition, and Speed

Originally a thread on X/Twitter:

Making decisions when faced with incomplete information is what every Entrepreneur has to contend with on a daily basis. The same is true for Investors. Curious what role Intuition plays?

Curious how Intuition is misunderstood and misused? Read on:

2/22: Making a decision when complete information is available isn’t terribly difficult. Outcomes aren’t guaranteed, but it’s generally true that the more information one has about a situation the easier it will be to map out a productive path forward.

3/22: Of course there’s no such thing as “complete information” so decision makers have to guide their organizations armed only with limited and imperfect information. Great Founders and Investors build conviction and act decisively even when faced with uncertainty.

4/22: Roosevelt was known for acting decisively when faced with uncertainty. He brilliantly summed up his own philosophy:

“In any moment of decision the best thing you can do is the right thing, the next best thing is the wrong thing, and the worst thing you can do is nothing.”

5/22: So when information is incomplete, does intuition act as the missing link in building conviction and acting decisively? The answer is both “yes” and “no” because intuition is often confused with a variety of not-so-desirable-traits that can be counterproductive.

6/22: Let’s start by defining intuition:

“Intuition is the ability to understand something immediately, without the need for conscious reasoning.” Some describe it as trusting your gut. But this is far too simplistic and a crutch that’s often used to justify sloppy thinking.

7/22: Intuition is a quickly generated point of view about a particular situation or issue that’s derived from one’s experiences. Unlike structured thinking which is grounded in frameworks and logic, intuition exists in the unconscious.

8/22: Intuition gives us the ability to “know” something directly without going through the process of analysis, synthesis, framework development and logical conclusions. It links the conscious and unconscious parts of our mind and acts as a bridge between instinct and reason.

9/22: If Entrepreneurs are to succeed in today’s rapidly changing environment, decisions need to be made with alacrity. But speed doesn’t allow for “complete analysis” which is why intuition matters. And the more innovative an idea is the less data there is to ground decisions.

10/22: And to be clear, scientific logic doesn’t have to be rejected in order to value intuition. Our name might make one think that we don’t appreciate instinct (QED = Quod Erat Demonstrandum), but even those of us at QED know that seeking balance is the optimal destination.

11/22: A great analogy can be made using Regression Analysis as the focal point.

Regression analysis attempts to determine the strength of a relationship between an outcome variable and a series of other variables. It’s an amazing statistical tool used to guide decision makers.

12/22: More data is better than less data because the data is used to “fit” a curve that describes how a system works. The goal is to pull out all the relevant signal and “know” with high confidence how stimulus X will impact result Y.

13/22: But the startup world doesn’t always have a lot of data to work with. Data is typically scarce when new and innovative products, experiences and business models are in the process of being built.

Less data = Less ability to rely on analytic conclusions as one’s Sherpa.

14/22: Imagine being asked to fit a Regression Curve to a very small data set. It’s uncomfortable and not terribly valuable.

Intuition is the equivalent of relying on a Regression Curve based on a single data point where the Curve might not even go through the data point.

15/22: Does this mean that intuition dominates structured thinking? Are we in a think fast act fast world? Should Entrepreneurs and VCs rely on instincts more and data less?

Yes and no.

Data and analysis are extremely valuable AND intuition is a skill that many lack.

16/22: When data exists, it would be silly to ignore listening to what it tells us. We live in a world where data is everywhere and easy to manipulate. Data should always be an input to our decision making processes because it’s a rarity that precisely zero data exists.

17/22: The danger with intuition is that for certain people it can be a poor guide. This happens when:

– Their intuition is actually disguised bias

– Their intuition has been shaped by a limited experience set

– Their intuition is really hasty generalization

18/22: Disguised Bias

The unconscious taps into past experiences to generate conclusions in a nonlinear way. The “how” and the “why” aren’t easily understood. This can lead to personal bias affecting decisions. Reducing unconscious bias is not easy to do because it hides well.

19/22: Limited Experience

People with limited life experience don’t know whether their intuition is good or bad. It takes both positive and negative experiences to guide intuition-based conclusions. A giant mistake is trusting your intuition before it’s been tempered by life.

20/22: Hasty Generalization

Hasty generalization is a logical fallacy that assigns too much weight to conclusions based on limited data. Small N = Less Signal. Pulling too much out of less signal is a recipe for disaster. Small N often demonstrates bias that drops out later.

21/22: The truth is that we need both instinct and reason to make the best possible decisions as Entrepreneurs and VCs. Many of us are uncomfortable using intuition as a guide and are embarrassed to say that we follow hunches.

22/22: The key is getting the balance right. The conscious mind is a logical machine and should be used relentlessly when real data and facts are present. Intuition plays a critical role when real data and facts aren’t present but only when one’s intuition is truly grounded.