Three Great New Columns on Avoiding Strategic Failure

The first is "Anomaly Detection: The Art of Noticing the Unexpected". Dr. Gary Klein.

With our consulting work with clients at Britten Coyne Partners, and in our Strategic Risk Governance and Management courses at the
Strategic Risk Institute, we emphasize being alert to the feeling of surprise, and writing down what caused it, as a powerful method for identifying emerging threats and avoiding failure.

Writing down surprises is critical, because, as Daniel Kahneman explained in
Thinking Fast and Slow, our mind’s automatic System 1 reasoning will quickly attempt to fit the surprise into our existing mental models and beliefs. When this happens, they feeling of surprise and our memory of what triggered it will both disappear – unless we write it down, to enable conscious System 2 to think about what it could mean.

Klein notes that, “An anomaly is a violation of our expectancies that enables us to revise the way we understand a situation…Most deviations and outliers are uninteresting. At the cognitive level, anomalies matter primarily when they have the potential to alter the way we understand a situation. And that type of sensemaking is very different from the flagging of outliers found in statistical methods.”

This brings us to Bent Flyvbjerg’s new paper, “
The Law of Regression to the Tail.” Fyvbjerg makes a point we have frequently made over the years: While that introductory statistics course firmly implanted the normal (Gaussian or Bell Curve) distribution in our mind, it is actually a very poor description of the distribution of outcomes that are typically produced by the complex adaptive systems that surround us (e.g., product markets, industry dynamics, the economy, politics, war casualties, etc.). Instead of the Bell Curve, complex adaptive systems produce outcomes that are much better described by power laws.

As Flyvbjerg notes, these distributions “have no population mean, or the mean is ill defined due to infinite variance. In other words, mean and/or variance do not exist. Regression to the mean is a meaningless concept for such distributions, whereas what one might call ‘regression to the tail is meaningful and consequential.”

What people under the spell of the normal distribution fail to realize is “We live in the age of regression to the tail. Tail risks are becoming increasingly important and common because of a more interconnected and fragile global system of human interaction… The pandemic and the climate crisis are presently the two most significant manifestations of the law and age of regression to the tail.”

In his new
Financial Times column, “The Power of Negative Thinking”, Tim Harford reminds us of the importance, in a world characterized by power laws and tail risks, that “we should all spend more time thinking about the prospect of failure and what we might do about it. It is a useful mental habit but it is neither easy nor enjoyable.”

This is a point also made by Gary Klein, who more than anyone has popularized the use of Pre-Mortem analysis, a method whose efficacy we have seen demonstrated time and again in our work with Britten Coyne Partners’ clients.

Best of all, it is often relatively quick and easy to apply. Assume it is some point in the future, and your initiative or strategy or start-up has failed. Ask the members of your team to anonymously write down their answers to three questions: (1) Why did we fail? (2) What warning signs did we miss? (3) What could we have done differently to avoid failure? Collect the answers, type them up, then collage, print, and distribute them back to the team. I have never seen a resulting discussion that did not produce a much better (and less risky) plan.

As Harford notes, “if we expect that things will go wrong, we design our projects to make learning and adapting part of the process. When we ignore the possibility of failure, when it comes it is likely to be expensive and hard to learn from.”

As we enter a period of unprecedented uncertainty, the likelihood of failure has exponentially increased.

The good news is that there are methods you can learn and apply that will improve your (and your organization’s) ability to anticipate emerging threats, appropriately assess them, and adapt to them in time to avoid strategic failure.


T
om Coyne and Neil Britten co-founded Britten Coyne Partners and the Strategic Risk Institute LLC, which provide consulting and education services that enable clients to successfully meet strategic risk governance and management challenges.

Comments

What's Ahead for the Economy? Insights from the KC Fed's Jackson Hole Symposium

At Britten Coyne Partners, the Strategic Risk Institute, The Index Investor, and The Retired Investor, our goal is to help clients avoid strategic failure and the painful losses it brings.

Our core process for accomplishing this goal is shown in the chart below. We stress the importance of anticipating and monitoring of emerging threats, and being alert to surprises that often indicate a new threat you have missed. We also stress the importance of appropriate assessment, early warning, and adapting in time using multiple approaches to minimize the impact of dangerous threats.


H2 Avoid Failure Chart


With this model in mind, I always pay attention to the academic research presentations that are on the agenda for the Federal Reserve Bank of Kansas City’s annual Jackson Hole Symposium (colloquially known as summer camp for the world’s most important central bankers).

This year's conference opened today, and the two papers featured this morning were on issues we have frequently addressed at BCP, SRI, Index, and Retired.

The first paper was “
What Happened to U.S. Business Dynamism?” by Ufuk Akcigit and Sina Ates. The authors note, “Market economies are characterized by the so-called “creative destruction” where unproductive incumbents are pushed out of the market by new entrants or other more productive incumbents or both...

“A byproduct of this up-or-out process is the creation of higher-paying jobs and reallocation of workers from less to more productive firms. [However], the U.S. economy has been losing this business dynamism since the 1980s and, even more strikingly, since the 2000s. This shift manifests itself in a number of empirical regularities", which Akcigit reviewed at this morning's session:

1. Market concentration has risen.

2. Average markups have increased.

3. Average profits have increased.

4. The labor share of GDP has gone down.

5. Market concentration and labor share are negatively associated.

6. The labor productivity gap between frontier and laggard firms has widened.

7. Firm entry rate and the share of young firms in economic activity has declined.

8. Job reallocation has slowed.

9. The dispersion of firm growth has decreased.

10. Aggregate productivity growth has fallen, except for a brief pickup in the late 1990s.

11. A secular decline in real interest rates has occurred.

Akcigit and Sina Ates’ observations are also consistent with research from McKinsey, which found that, “the top 10 percent of companies now capture 80 percent of positive economic profit…[Moreover], after adjusting for inflation, today’s superstar companies have 1.6 times more economic profit, on average, than the superstar companies of 20 years ago” (“
What Every CEO Needs to Know About Superstar Companies”).

Of the hypotheses that Akcigit and Ates tested to explain these trends, they found the evidence and their modeling best supported the hypothesis that, “reduction in knowledge diffusion [across firms] between 1980 and 2010 is the most powerful force in driving all of the observed trends simultaneously.”

Discussion at this morning’s symposium focused on the plausible obstacles to faster diffusion of advanced knowledge across firms. These included more patenting by larger firms, larger firm’s acquisition of patents from smaller firms, aggressive patent litigation by large firms, large firms luring away employees with the most patents from smaller firms, and larger firms’ heavy investment in lobbying and supporting regulatory changes that strengthen their advantage.

I was surprised, however, that another very likely obstacle to faster diffusion wasn’t mentioned this morning. In “
Digital Abundance and Scarce Genius”, Benzell and Brynjolfsson found that the shortage of talented employees is the most important constraint on the faster deployment and diffusion of advanced technologies across the economy. And Korn Ferry found, in “The Global Talent Crunch”, that “the United States faces one of the most alarming talent crunches of the twenty countries in our study”.

So what is to be done, given the authors’ observation that the COVID-19 pandemic will likely make these conditions worse?

Looking at possible policy changes that could help to avert this outcome, this morning’s discussion focused on the need for stronger anti-trust enforcement and other actions that would intensify the level of competition in the US economy. To these I would add that recovering students’ COVID-19 learning losses and substantially strengthening the US education system are also critical (and will require painful structural changes, not just further infusions of cash).

The second paper presented this morning was “
Scarring Body and Mind: The Long-Term Belief Scarring Effects of COVID-19”, by Kozlowski, Veldkamp, and Venkateswaran.

They find that, “the largest economic cost of the COVID-19 pandemic could arise from changes in behavior long after the immediate health crisis is resolved. A potential source of such a long-lived change is scarring of beliefs, a persistent change in the perceived probability of an extreme, negative shock in the future…

“The long-run costs for the U.S. economy from this [belief] channel are many times higher than the estimates of the short-run losses in output. This suggests that, even if a vaccine cures everyone in a year, the Covid-19 crisis will leave its mark on the US economy for many years to come.”

This is consistent with Robert Barro’s earlier research on the impact of “disaster risk” on investors’ decisions and required returns (see his 2006 paper on “
Rare Disasters and Asset Markets in the 20th Century).

It is also consistent with the findings in another recent paper, “
The Long Run Consequences of Pandemics”, by Jorda et al from the Federal Reserve Bank of San Francisco.

They analyzed the medium to long-term effects of pandemics, and how they differ from other economic disasters, by studying major pandemics using the rates of return on assets stretching back to the 14th century.

They concluded that, “significant macroeconomic after-effects of pandemics persist for decades, with real rates of return substantially depressed, in stark contrast to what happens after wars”, and observe that “this is consistent with the neoclassical growth model: capital is destroyed in wars, but not in pandemics; pandemics instead may induce relative labor scarcity and/or a shift to greater precautionary savings” by altering consumer’s beliefs.

This morning’s discussion of the paper by Kozlowski et al focused on the critical question of why belief scarring seemed to have had a much stronger and longer-lasting impact after the Great Depression than after the 9/11 terrorist attacks.

The consensus seemed to be that a range of very visible policy responses to reduce the risk of further terrorist attacks after 9/11 seemed to reduce belief scarring by much more than the policy responses to the Great Depression..

I
n sum, along with actions to restore business dynamism and strengthen competition, public perceptions of the efficacy of various policy responses to the COVID-19 pandemic will very likely be critical to minimizing its long-term negative impact on economic activity. Both of these are key indicators to monitor in the months ahead.



Britten Coyne Partners advises clients on strategic risk governance and management issues. The Strategic Risk Institute provides online and in-person courses leading to a Certificate in Strategic Risk Governance and Management. Since 1997, The Index Investor has published global macro research and asset allocation insights, with a particular focus on avoiding large portfolio losses. The Retired Investor has the same focus, customized for the unique needs of investors in the decumulation phase of their financial life.


Comments

How to Avoid Intelligence Analysis Errors

This week, Mike Morrell had former senior CIA executive Martin Peterson on his Intelligence Matters podcast, which is well worth listening to.

Petersen succinctly summarized the lessons he'd learned over a career (and taught to new analysts) about the root causes of many intelligence analysis errors. They apply equally well to many situations outside the world of intelligence, where analysts and decisions makers must make sense of highly uncertain situations.

Petersen highlighted three classic types of error, and the questions to ask to avoid them.

(1) You don't have a good understanding of the organization you're trying to analyze.

  • How do you get to the top in this organization?
  • What is the organization's preferred method of exercising power and making decisions?
  • What are acceptable and unacceptable uses of power in this organization?

(2) You don't have a good understanding of the individuals making decisions.

  • How do they assess the current situation?
  • How do they see their options?
  • What is their tolerance for risk, under the current circumstances?
  • What do they believe about your capabilities, intentions, and will?
  • What is their definition of an acceptable outcome?

(3) You don't understand your own analysis.

  • Rather than asking someone how confident they are in their analysis, ask them where their analysis is most vulnerable to error. This is the same approach as the one we use, which is identifying the most uncertain assumptions in an analysis, and the implications of different outcomes for them. The underlying issues are also surfaced by use of pre-mortems, which we also recommend.
  • What are you not seeing that you should be seeing if your hypothesis/theory/line of analysis is correct? As Sherlock Holmes (and Thomas Bayes) both teach us, sometimes the dog that doesn't bark provides the most important evidence.
  • Petersen emphasized that if you ever catch yourself saying, 'It makes no sense for them to do that', it is a clear warning sign that you either don't understand either the organization and/or the decision makers that are the target of your analysis.

All important lessons to keep in mind as you try to make sense of the complex, highly uncertain, and fast changing situations that abound in the world we face today.

Comments

Using Narratives to Make Sense of High Uncertainty

The arrival of the COVID19 pandemic has made the distinction between risk and uncertainty painfully clear. In the case of the former, the range of possible future outcomes is known, as are their probabilities, and potential impact.

In the case of uncertainty, some or all of these are unknown.

We have many tools available to help us make good decisions in the face of risk. While the crises occasionally remind us that these tools aren’t perfect (e.g., Long Term Capital Management in 1998, the housing collapse in 2008, and COVID19 in 2020), most of the time they serve us well.

But that is not true when we confront highly uncertain systems and situations.

To be sure, our first reaction is to try to adopt our risk tools to help us in these situations. In place of probabilities based on the historical frequencies at which common events (e.g., car accidents) occur, we take the Bayesian approach, and substitute probabilities signifying our degree of belief that historically rare or unprecedented events will occur in the future.

The four years I spent on the Good Judgment Project reminded me once again of the benefits (e.g., superior forecast accuracy) that arise from the disciplined application of Bayesian methodology.

Yet over a forty-year career of making decisions in the face of uncertainty, I’ve also seen that Bayesian methods can sometimes creates a false – and dangerous – sense of security about the precision of our knowledge, leading to overconfidence and poor decisions.

What we need is broader mix of methods for analyzing and making good decisions under conditions of uncertainty (and ignorance, or “unknown unknowns”), not just risk.

With this in mind, I was excited to read a new paper, “The Role Of Narrative In Collaborative Reasoning And Intelligence Analysis: A Case
Study
”, by Saletta et al, which significantly adds to our understanding of the role of narratives and how we construct them to help us make sense of highly uncertain situations

At Britten Coyne Partners, we have written a lot about the important, if too little understood, role that narrative plays in both individual and collective sensemaking and decision making under uncertainty.

Researchers have found that when uncertainty rises, evolution has primed human beings to become much more prone to conformity and to rely more on imitating what others are doing (so-called "social learning" or “social copying”).

Paradoxically, as uncertainty increases, people are more likely to become attracted to a smaller (not larger) number of competing narratives that explain the past and present and sometimes predict possible future outcomes.

In other words, as uncertainty increases the conventional wisdom grows stronger, even as it is becoming more fragile and downside risks are rising.

Today's hyperconnected socio-technical systems — including financial markets — are therefore more vulnerable than ever before to small changes in information that trigger feelings (especially fear) and behavior that spread quickly, and are then further amplified by algorithms of various types. The increasing result is sudden, non-linear change.

In recent years, the role of narratives in economic cycle and financial markets has increasingly been a subject of academic inquiry (e.g., “Narrative Economics”, by Bob Shiller; “Constructing Conviction through Action and Narrative: How Money Managers Manage Uncertainty and the Consequences for Financial Market Functioning”, by Chong and Tuckett; and “News and Narratives in Financial Systems: Exploiting Big Data for Systemic Risk Assessment”, by Nyman et al).

While there are many definitions of “narrative” (and synonyms for it, like “analytical line”, and sometimes “mental models”), most have some common elements, including descriptions of context and key characters, actions and events that move the narrative forward through space and time, and causal links to outcomes and various types of consequences (e.g., cognitive, affective, and/or physical).

Saletta and his co-authors take our understanding of narrative from the macro to the micro level, and describe how, “individuals and teams use narrative to solve the kinds of complex problems organizations and intelligence agencies face daily.”

They “observed that team members generated “micro-narratives”, which provided a means for testing, assessing and weighing alternative hypotheses through mental simulation in the context of collaborative reasoning…

“Micro-narratives are not fully developed narratives; [instead] they are incomplete stories or scenarios…that emerge in an unstructured manner in the course of a team’s collaborative reasoning and problem solving...

[Micronarratives] “serve as basic units that individuals and teams can debate, deliberate upon, and discuss in an iterative process in which micro-narratives are generated and weighed against each other for plausibility with regard to evidence, general knowledge about the world, and fit with other micro-narratives. They can then be organized and assembled into a larger, more developed narrative…

The authors document that the intelligence analysts they studied “ran mental simulations to reason about evidence in a complex problem... [and] test and evaluate many diverse, interacting and often competing micro-narratives that were generated collaboratively with other team members...They tested the plausibility of these micro-narratives against each other, what they knew, and their best estimates (or guesses) for what they didn’t know”…

“In a non-linear and iterative process, the analysts used the insights developed in the process of generating and evaluating micro-narratives to develop a macro-level narrative.”

The authors conclude that, “narrative thought processes play an important role in complex collaborative problem-solving and reasoning with evidence…This is contrary to a widespread perception that narrative thinking is fundamentally distinct from formal, logical reasoning.”

This is also a very accurate description of the team forecasting process that I experienced during my years on the Good Judgment Project.

At Britten Coyne Partners, we also stress that this basic process of collaborative sensemaking in highly uncertain systems and situations can be further enhanced through the use of three complementary processes.

The first is structuring sensemaking processes around the three critical questions first described by Mica Endsley twenty-five years ago:

(1) What are the key elements (e.g., characters, events, etc.) in the system or situation you are assessing, over the time horizon you are using?

(2) What are the most important ways in which these elements are related to each other (e.g., causal connections and positive feedback loops that give rise to non-linear effects)?

(3) Given the interaction of the critical trends and uncertainties you have identified, how could the system/situation evolve in the future, either on its own or in response to actions you and/or other players could take?

The second is using explicit processes to offset what Daniel Kahneman has called the WYSIATI phenomenon (“What You See Is All There Is”), or our natural tendency to reach conclusions only on the basis of the information that is readily at hand. As Sherlock Holmes highlighted in “The Hound of the Baskervilles”, it is often the dog that doesn’t bark that provides the most important clue.

In “Superforecasting”, Professor Phil Tetlock showed how the WYSIATI problem can be overcome (and forecast accuracy improved) by combining the information at hand with longer term, “base rate” or “reference case” data.

Another approach is Gary Klein’s “pre-mortem” technique. Tell your team to assume it is some point in the future and their assessment or forecast has turned out to be wrong. Ask them to write down the information they missed or misinterpreted, including important information that was absent.

A final technique is to have your team assume that their original evaluation of the evidence was wrong, and to generate alternative assessments of what it could mean.

The third process is Marven Cohen’s critiquing method, which focuses on finding and resolving three problems that reduce the reliability of macro narratives (e.g., more formal scenarios). Incomplete narratives are either missing key narrative elements or represent them using assumptions rather than direct evidence. The second problem is the use of assumptions to explain away conflicts between available evidence. And the third problem is the use of doubtful assumptions that have weak evidential support.

In the post-COVID19 world, the ability to make sense of unprecedented uncertainty and maintain ongoing situation awareness under rapidly evolving conditions will be a hallmark of high performance teams.

Unfortunately, this is not a capability that has been developed in the course of many leaders’ previous training and experience. Mastering new tools and processes, like the use of narrative, is critical to organization’s future success.



These and other processes and tools for making good decisions in the face of unprecedented uncertainty are covered in our new online course, leading to a Certified Competence in Strategic Risk Governance and Management. You can learn more about it at the Strategic Risk Institute LLC (an affiliated of Britten Coyne Partners and Index Investor LLC).

A 50% discount on the launch price is offered for the first 100 subscribers. Enter the code 50PERCENTPOUND (for payment in pounds) or 50PERCENTDOLLAR (for payment in dollars) on the payment page.



Comments

Britten Coyne Announces New Online Version of their Strategic Risk Governance and Managment Certified Competence Course

Britten Coyne Partners and their affiliate, the Strategic Risk Institute LLC have launched an on-line programme leading to a Certified Competence in Strategic Risk Governance and Management.

This 12 module on-line course draws upon Britten Coyne Partners’ five years’ research and leading edge thinking. It qualifies for 40 hours of CPD.

“Overall I have enjoyed the material enormously and would not hesitate to recommend the course to anybody with a serious interest in this critical subject.” UK Chartered Director

"To undertake this learning whilst my company was hit by the Covid-19 pandemic gave the course an unnerving backdrop. But as well as discomfort I had an increasing array of tools and thinking at my disposal to use in a real life situation. The test of any tool is how it performs under pressure and this course delivered.” UK Chartered Director

"I graduated with a degree in history, and took this course to increase my appeal to employers. I found it to be rigorous, engaging, and deeply relevant to my future." Recent University Graduate


******* Enrollment now open *******

******* 50% “Early Bird” discount *******

A 50% discount on the launch price is offered for the first 100 students.

Go to the Strategic Risk Institute website for more information about the course and the enrollment link. Enter the code 50PERCENTPOUND (for payment in pounds) or 50PERCENTDOLLAR (for payment in dollars) on the payment page.
Comments