On Overconfidence and Cognitive Fallacy

In this week’s New York Times Magazine, Daniel Kahneman, emeritus professor of psychology and of public affairs at Princeton University, writes about the hazards of overconfidence. His piece begins with an anecdote from his days in the Israeli army and then moves on to lambasting overconfidence of professional investors. A lot of the ideas here I was already familiar with, but I liked Kahneman’s conclusion:

We often interact with professionals who exercise their judgment with evident confidence, sometimes priding themselves on the power of their intuition. In a world rife with illusions of validity and skill, can we trust them? How do we distinguish the justified confidence of experts from the sincere overconfidence of professionals who do not know they are out of their depth? We can believe an expert who admits uncertainty but cannot take expressions of high confidence at face value. As I first learned on the obstacle field, people come up with coherent stories and confident predictions even when they know little or nothing. Overconfidence arises because people are often blind to their own blindness.

True intuitive expertise is learned from prolonged experience with good feedback on mistakes. You are probably an expert in guessing your spouse’s mood from one word on the telephone; chess players find a strong move in a single glance at a complex position; and true legends of instant diagnoses are common among physicians. To know whether you can trust a particular intuitive judgment, there are two questions you should ask: Is the environment in which the judgment is made sufficiently regular to enable predictions from the available evidence? The answer is yes for diagnosticians, no for stock pickers. Do the professionals have an adequate opportunity to learn the cues and the regularities? The answer here depends on the professionals’ experience and on the quality and speed with which they discover their mistakes. Anesthesiologists have a better chance to develop intuitions than radiologists do. Many of the professionals we encounter easily pass both tests, and their off-the-cuff judgments deserve to be taken seriously. In general, however, you should not take assertive and confident people at their own evaluation unless you have independent reason to believe that they know what they are talking about. Unfortunately, this advice is difficult to follow: overconfident professionals sincerely believe they have expertise, act as experts and look like experts. You will have to struggle to remind yourself that they may be in the grip of an illusion.

My only gripe: I believe overconfidence is even more systemic than Kahneman posits. I believe there are overconfident dentists, waitresses, accountants, and engineers. What about you?

One thought on “On Overconfidence and Cognitive Fallacy

  1. Yes. I project a confidence that exceeds my true sense of self and my true abilities. Even overly-confident people have self-doubt. (I believe there are some who don’t have self-doubt, even in their quiet moments of self-reflection; but I expect that would be a minority.)

    I suppose that we should discuss what we mean by overly-confident and whether it makes sense to distinguish the degree or extent of the over-confidence. For instance, a megalomaniac’s over-confidence is extreme and pathological. On the other hand, a doctor who feels a small degree of over-confidence might use that confidence to give himself or herself endurance to handle complex situations.

    Another example is if a business person without over-confidence meets economic adversity, that person might be more likely to give up than a business person with over-confidence. The business person with over-confidence will be more apt to persist because of their greater belief in their capability, and not to capitulate to the adversity.

    When we think of the personality characteristic of ‘persistence’ we tend to think of that as a positive character-trait. But it is easy to consider over-confidence as a negative characteristic. I think the reason for that is that a high degree of over-confidence comes across as arrogance. We view arrogance as a negative trait. And to the extent that overly-confidence is connoted with arrogance, over-confidence seems undesirable. So we should be considering to what degree of over-confidence it is that we are talking about.

    On the other hand, if we are talking about the negative quality of ‘persistence’ we are more likely to use the term ‘stubborn’ rather than the word ‘persistence’. So we don’t tend to connote persistence as a negative characteristic even though stubbornness is an extreme degree of persistence.

Leave a comment