Skip to content

Making Smarter Decisions: 7 Lessons from the CIA for Founders & Investors

Social Sharing

09.23.2024

Kathryn Weinmann

Principal, FirstMark

Imagine you are a CIA analyst evaluating the likelihood of attacks on the US. Even with incomplete and ambiguous intel, you are required to provide a clear perspective for life or death decisions. Despite being smart and dedicated to protecting the country, your approach to the analysis is likely flawed.

It’s not your fault – our brains have some major limitations in how we process information and make judgment calls. Fortunately, we can overcome some of these biases with the help of the CIA. Cognitive psychology expert and intelligence veteran Richards J. Heuer wrote The Psychology of Intelligence Analysis with tactical advice for all decision makers.

While venture investors and founders are not (usually) dealing with state secrets, we face similar challenges as CIA analysts. Cognitive biases impact VC decisions, sometimes with billion dollar consequences. One top fund initially passed on Airbnb, believing people would not rent out their homes to strangers. Another thought Uber was too niche, given the size of the black car market. Every firm has versions of these mistakes, but the best firms learn and build processes to avoid them in the future. Thoughtful founders have it even harder – they must not only be intellectually honest in running their business but also translate that into a compelling message for investors. Understanding the brain’s wiring can help investors make better decisions and help founders navigate obstacles in the fundraising market. 

Here are seven lessons with tactical suggestions to help anyone make smarter decisions.

1. Memory is Subjective: Why Experience Can Be a Double-Edged Sword

Long term memory has relatively unlimited capacity, but it’s not as reliable as we think. Memories are patterns of connections between neurons – the more frequently the path is followed, the stronger it becomes. As a result, who observes something influences what is observed.

For example, chess grandmasters’ years of experience enable them to assess an active game of chess almost immediately. Even after a quick glance, grandmasters can recall the positions of most pieces, while non-chess players struggle to remember more than a handful.[1]

That said, these pathways can become mental ruts, making it difficult to reorganize and correctly take in new information. Whenever you feel like a grandmaster in an area of life, before asserting that you have “seen this movie before,” consider that, in the real world, the rules are rapidly changing. 

Snapchat was initially dismissed by multiple prominent investors, including ones who knew social media well. The appeal of disappearing photos was lost on a generation who assumed building a social media platform required a content history that users built over time.

Similarly, many investors who looked at Stripe’s early rounds were fintech investors and experts in the payments space. Precisely because they were grandmasters, they dismissed Stripe’s potential within what they believed to be a crowded space. Our experience is as likely to work against us as it is for us, so it is important to actively seek alternative interpretations of what could be going on.

Instead, try this: Whatever you expect based on experience, consider the opposite and explore how that could be true. Try role playing, assigning yourself a different perspective than your first inclination.

2. First Impressions Matter: It’s Hard to Unsee

Especially when dealing with people who see themselves as experts, first impressions matter. It’s hard to “unsee” something, even with new information. 

Heuer shares a study where participants were shown the below series of drawings, each slightly altered from the previous image to transform from a man’s face into a woman’s form.[2] Take a look starting from the left – what do you see? As you move to the right, when do you see the woman’s form appear? Now start from the right and move left. When do you see the man’s face appear?

What you see may depend on where you started. For better or worse, perspective gets locked in – even new, conflicting information gets assimilated. As a result, gradual change tends to go unnoticed.

When we first see something ambiguous, we try to interpret it. However, the longer we see the blurred image, the more confident we get in our interpretation; we construct our own reality. So even after the blur is reduced and we can see more, we have a hard time changing our minds. 

Once we form an opinion, we tend to need way more information to invalidate a hypothesis than we needed to form it initially. For founders, that means remembering that people will fill in the blanks on their own, stressing the importance of knowing what information gaps are left open. For example, if you are creating a new market, say so – don’t let investors fill in the blank about market size.

Instead, try this: Investors, document your initial perspectives and reasoning. For longer deal processes, bring in fresh perspectives to challenge first impressions and note what information could change your mind. Founders, be mindful of what gaps are left open – investors will fill in the blanks. Ask investors their view on the space to get ahead of concerns before they crystalize.

3. Think Outside the (Invisible) Box: Breaking Mental Patterns

We learn to connect the dots in a certain way, making it difficult to think outside the box. Consider this exercise. Draw no more than four, straight, contiguous lines that cross through all nine dots below:[3] 

Hint: If you feel stuck, consider talking out loud. The human brain processes written and spoken language in different areas, so it can be a helpful way to prompt yourself to think differently.[4]

There are several ways to solve the puzzle, two of which you can see here.

Most people create an imaginary box around the nine dots, so they don’t think outside it. Others forget that the lines can intersect. Others still assume the lines must pass through the center of the dots. If you look at either of the answers and reflexively think, “I could have gotten that,” beware of hindsight bias that makes things look easier in retrospect.

Investors face similar risks, often rushing to apply successful playbooks to their next investment or underestimating the potential in tough spaces. Many initially passed on Pinterest, doubting its business model. However, Rick (FirstMark Co-founder & GP) could see the paradigm shifting. He knew the future of the internet would be visual and would require curation on mobile, giving him the confidence to invest at the Pre-Seed round when Pinterest was still an idea. Other investors overlooked Shopify, dismissing the potential to build outside of Silicon Valley. Amish (FirstMark Co-founder & GP) saw what blogging was doing to publishing and saw a similar opportunity in commerce, allowing him to invest behind a generational trend: the broad democratization of entrepreneurship.

Pattern matching alone is overstated. Often cited as the Holy Grail of venture analysis, it can lead to reductive thinking. Our brains auto-process new information, so what can feel like intuition is really a product of unquestioned assumptions. Even knowing this risk does not stop it, so we must consciously raise our level of critical thinking.

Unfortunately, investors often cite ‘pattern matching’ when they mean a simplistic ‘analysis by analogy’. Analogies are tempting, particularly when personal. This is a constant challenge for founders, who must pitch their company as fitting within established patterns of success while also conveying the right amount of differentiation.

Instead, try this: Use analogies to highlight differences between situations and to generate new hypotheses rather than to draw conclusions. Founders, help investors see ‘what’s different here’ and identify the assumptions holding others back in your space.

4. Don’t Rush to Judgment: Understand Your Risk  

Both intelligence circles and VCs push for premature closure. In intelligence, analysts are pressured for a perspective within 2-3 days of an event. New information tends to get interpreted through the same lens. Similarly, investors often conflate clear thinking with a conclusive point of view.

In deal memos, words like “likely”, “probably”, and “could” are vague – you might mean 90% likely, and I might mean 51%. While it’s impossible to handicap exact percentages, analysts and investors shouldn’t hide behind unclear language.

Our brains struggle with accurately assessing probabilities. We remember occurrences more than non-occurrences, and we tend to overestimate the likelihood of repeat events. People estimate likelihood based on memory and then adjust with new information. Often, though, we don’t adjust enough.

Instead, try this: Both investors and founders should outline and challenge assumptions. Be specific about uncertainties and try to estimate percentages – not for accuracy, but for clearer thinking.

5. Seek to Disprove, not Prove: Getting to the Right Answer

Getting to the right answer in analysis is like the process of medical diagnosis – use symptoms to create hypotheses, then conduct tests to rule out the wrong ones. One piece of evidence disproving a hypothesis is more conclusive than 100 pieces supporting it. Counterintuitively, the most likely hypothesis is often the one with the least evidence against it, not with the most for it.

There are countless ways startups fail. Early-stage businesses rarely show strong evidence of probable success (>50%). Even when investors say “yes,” they often do so knowing failure is still the most likely outcome. This is why investors put so much effort behind understanding the upside attached to even a small chance of success.

An often-overlooked hypothesis is deception – you might be getting played. Founders walk a tightrope of building the plane while they fly it while avoiding misleading investors or engaging in full-fledged fraud. Reputational risk acts as a safeguard, but an active search for evidence is needed to feel comfortable ruling out the deception hypothesis (though you can never prove it isn’t happening).

Instead, try this: Focus on disproving hypotheses. Use sensitivity analyses to identify how key variables impact your thesis. Assign devil’s advocates or define what new information would change your mind, then seek it out. Write a pre-mortem: If the investment failed, what likely happened? Identify milestones for future observation and leave an audit trail so there’s an answer for: “What were they thinking?!”. Occasionally, re-examine the problem from scratch to avoid incrementalism. Encourage dissent and seek fresh perspectives, especially from junior voices. Pay attention to feelings of surprise – when something doesn’t add up, find out why.

6. Less is More: Why More Data Can Lead to Worse Decisions

Analysts often don’t realize their judgments are shaped by a few dominant factors rather than all information. Heuer notes that “Intelligence failures are usually caused by failures of analysis, not failures of collection.”[5]

To illustrate the point, horse race handicappers were asked to predict a race using increasing amounts of variables. They predicted the outcome of a race 4 times – first using their top 5 variables, then 10, 20, and 40.[6] While their accuracy remained the same across the 4 predictions, the handicappers’ confidence rose steadily with the additional information used. More data made them overconfident. Perhaps more alarming, the same findings were found with clinical psychologists tasked with assigning a diagnosis.[7]

In venture, this shows up when investors overweight correlated but not diagnostically valuable data (e.g., team size, press mentions, amount fundraised). The human brain has an oversensitivity to the consistency of evidence, so it takes concerted effort to emphasize credibility and value.

Instead, try this: Pause your analysis to determine what questions remain and how new information would impact your confidence. Identify which data points could be noise rather than signals.

7. Beware of Hindsight Bias: What You Knew vs. What You Know

It all seems obvious looking back. Analysts overestimate the accuracy of their past predictions. Those who read intelligence reports underestimate what they learned from those reports. After an event occurs, many claim they “knew it all along.” Events often seem more predictable in retrospect than they actually were.

Looking back, what is relevant is clearer, and we forget what we knew before the event occurred. In one study, participants estimated the likelihood of each of 4 potential outcomes in the British-Gurkha conflict.[8] Only one group was told the true outcome but were still asked to estimate the likelihood of each outcome as if they did not already know. The group that knew the outcome had approximately a 2x higher probability placed on the outcome that indeed occurred.

Venture firms need institutionalized processes for comparing judgements with outcomes, not to assign blame but rather to improve decision making. Instead of citing long feedback cycles, focus on viewing recent events against the assumptions and hypotheses outlined in the investment / pass decision. While exits are the ultimate metric, shorter time frames can better evaluate the quality of analysis. This helps distinguish good or bad judgment from good or bad luck. It fosters the conditions for repeatability and for institutional rigor as firms train younger investors.

Instead, Try This: Acknowledge that you can’t fully recreate unknowns looking back. However, you can ask the counterfactual: “If the opposite had occurred, would I have been surprised? Would that outcome have been predictable?”

Know Thyself

The key is knowing how our brains process information. That includes knowing our own values and assumptions. By examining how we think rather than just how we communicate, we can make the right call more often.

Of course, this is easier said than done. Investors rely on founders who are often master storytellers. Even still, cognitive limitations can cause great founders and ideas to be overlooked. The venture community needs to break the mold of how we identify winners – to look at companies better, we must first look more closely at ourselves.

Legendary founders always break the mold in some way. At FirstMark, we strive to do the same, which is what led us to invest in companies like Pinterest, Shopify, Airbnb, DraftKings, Ro, and Discord. With our latest raise of $1.1 Billion, we are backing founders building the next generation of iconic companies. If that’s you, please reach out.

 

 


[Solution to the dots challenge]

          

[1] George A. Miller, “The Magical Number Seven—Plus or Minus Two: Some Limits on our Capacity for Processing Information.” The Psychological Review, Vol. 63, No. 2 (March 1956).

[2] Drawings devised by Gerald Fisher in 1967.

[3] The puzzle is from James L. Adams, Conceptual Blockbusting: A Guide to Better Ideas. Second Edition (New York. W. W. Norton, 1980), p. 23.

[4] Jerry E. Bishop, “Stroke Patients Yield Clues to Brain’s Ability to Create Language.” Wall Street Journal, Oct. 12, 1993, p.A1.

[5] Richards J. Heuer, Psychology of Intelligence Analysis, 1999.

[6] Paul Slovic, “Behavioral Problems of Adhering to a Decision Policy,” Unpublished manuscript, 1973.

[7] Stuart Oskamp, “Overconfidence in Case-Study Judgments,” Journaling of Consulting Psychology, 29 (1965), pp. 261-265.

[8] Experiments described in this section are reported in Baruch Fischhoff, “Hindsight does not equal Foresight: The Effect of Outcome Knowledge on Judgment Under Uncertainty,” Journal of Experimental Psychology: Human Perception and Performance, 1, 3 (1975), pp. 288-299.