Harvard students comment on "Uncertain Risks"

I think the inclusion of the Deer Hunter and Russian Roulette analogies throughout the article prove to be helpful to non-quantitatively inclined readers. As is addressed early in the article, misunderstanding or confusing the words "risk" and "uncertainty" can be dangerous, and doing so is a key reason why many misunderstand COVID-19. I think the first paragraph of the article does a great job of defining these words through the easily-understood analogies of hunting a deer and Russian Roulette. The article addresses all of the variable factors involved in determining the odds of killing a deer in a single shot which perfectly defines uncertainty, and the blunt 1 in 6 chance of dying in Russian Roulette which defines the other extreme, nearly complete certainty. These examples clearly differentiate risk, the chance of death, from uncertainty, how sure we are about the amount of risk, which I think is really helpful for non-quantitatively inclined readers.

1. Which part(s) of the discussion do you expect would be most illuminating for non-quantitatively-inclined readers?

I really like how the article suggests we put uncertain events into four "buckets" - high-risk high-uncertainty (Apollo 11), high-risk low-uncertainty (Russian roulette), low-risk high-uncertainty (getting COVID-19), and low-risk low-uncertainty (driving). While this approach applies "stereotypes" to uncertain events, it's often much easier for humans to think about things when using categories as compared to the raw numbers.

2. Suggest another framing of the issues discussed that would be more effective.

One method I love for evaluating risk of death is the micromort along with confidence intervals. In keeping with conventions for metric units, one micromort is a 1/1,000,000 (one in a million) chance of death. For example, skydiving comes with a risk of 8 micromorts per jump (8 in a million chance of death), while traveling 230 miles by car comes with a risk of 1 micromort. This unit lets us convert risk from a hard-to-understand fraction into a nice, low, round number, which is much easier to comprehend.

Additionally, micromorts let us use error bars that make sense on our numbers. For example, skydiving has a risk of 8.4 micromorts per jump, with a 95% confidence interval between 7.7 micromorts and 9.3 micromorts - a low-risk low-uncertainty activity. But going to space, by contrast, we now know has risk of 32,000 micromorts (not taking into account modern advances in spaceflight technology), with a 95% confidence interval between 21,000 and 49,000 micromorts - far less certain than skydiving.

As someone who wasn't entirely sure of the difference between risk and uncertainty, I found the example of the gun and Russian roulette extremely enlightening in differentiating the uncertainty of death -- which is almost 0 in the case of a bullet entering the brain -- and risk, which is the probability that the bullet will be in the chamber fired; in this case, 1/6. I also appreciated the examples of other easily visualizable isntances of different uncertainty/risk tradeoffs, such as the living room couch for low risk/low uncertainty and space exporation (Apollo 11) for high risk and high uncertainty. These examples really helped flush out the concepts before they were applied to the low risk, high uncertainty scenario of COVID-19

As someone who has never taken a rigorous stem course at Harvard nor considered themselves a quantitatively inclined reader, this essay was surprisingly helpful and clear for me when it came to understanding the difference between risk and uncertainty. Even though the risk/uncertainty matrix professor Goodman supplied was a qualitative representation of the issue, the quantitative aspects of things like covid modeling help us understand why mitigation is important in cases of high uncertainty particularly due to the human factors at play. I thought the article blended these two perspectives really well through the numerical examples provided in the covid examples in particular.

I think for non-qualitatively inclined readers, the covid risk discussion is the most useful. Particularly, when it says that the average deaths from the United States are only estimated to increase by 10%. I think that most people think that a relatively low number of people die each year, and so just one more person dying per 10 estimated deaths in a normal year would not concern them greatly. As such, I think they would be able to understand the chance of dying from Covid is not necessarily that great, and they can view the risk with more of a probability assessment rather than a "might happen" assessment.

2. Another way I might choose to frame this discussion has to do with another way to think about uncertainty. Rather than think about where the “real odds” may be, one can think of it in terms of how much you should be willing to adjust your odds. For instance, one can be very certain in a prediction that a coin flip will be heads exactly 50% of the time. Because we can have high certainty, it would take a lot of information to change your opinion. If someone told you “they knew” heads only came up 30% of the time you wouldn’t believe them; and if you flipped a coin 10 times and got 8 heads it wouldn’t really change your opinion on the probability. But let’s say you have a weighted coin, but you don’t have any idea how it is weighted. Now if you flipped the coin 10 times and got 8 heads, you would believe that the coin is probably a little weighted towards the head. Thus, one can have a real understanding of what uncertainty in prediction means through a more application-based explanation.

I found this essay very engaging and provided the reader with commentary and discussion that could be more easily digested than trying to interpret lots of statistics and quantitative evidence about this subject. Personally, I would consider myself a non-quantitatively-inclined reader and find that my best understanding of topics comes from broken down discussion and interpretation that is focused on the topic rather than trying to interpret all of the numbers that support that topic. In this essay, I appreciated the lens that deconstructed the complexities of these ideas; even in the first paragraph, there was a quote from a famous movie that one might know and then some explanation that was connected to common known activities like hunting and Russian roulette. From that point, I already had a better grasp of the knowledge trying to be conveyed than if I had tried to read a daunting quantitatively focused essay. The fact that the Deer Hunter Russian Roulette story returned to the conversation to describe the differences in uncertainty and risk which was also very illuminating and grounded my understanding. The reader is then able to connect the examples at the beginning of the essay to how it all relates to COVID-19 and epidemiology, which was a great way to help the reader to best understand the basics of risk and uncertainty and then more easily apply it to their uptake of this knowledge surrounding the current pandemic.

I think using the example of the One Deer Hunter Russian roulette story throughout the explanation and then weaving different components of the topic to be understood using the same example is something that non-quantitatively inclined readers would find to be most helpful. Especially since it allows them to apply this knowledge to a real-world working scenario and view the different definitions of the risk and uncertainty can be more easily applied. The example is used in the first paragraph to compare more complex odds with many factors playing a role such as deer hunting to the easily calculate odds when playing Russian Roulette. By then using these same scenarios to explain low uncertainty and high experiences we are able to better understand the details behind them.

I believe the main takeaway of the essay for non-quantitatively-inclined readers should be the discussion of why consider uncertainty in the first place. Humans are not perfect and when we attempt to theorize, experiment, or evaluate observable phenomena there is no way our results will hold 100% of the time. However, having a notion of 'error-spread' or uncertainty is crucial to make this idea transparent and thus make better science and even everyday decisions.

The section that I expect would be most illuminating for non-quantitatively inclined readers would be the part two situations with different uncertainties. In this section it looks first at a low-uncertainty situation (Russian Roulette) then at a high uncertainty situation (Apollo 11). I think it is very important to apply these easily comprehensible examples towards two drastically levels of uncertainty because it makes it a much easier access point. Jumping straight into numbers with little context can lead to people just giving up and tuning out the rest of the information. However, using these tangible situations and very little, simple math it is much easier to get a foot in the door of this information. Once that ground layer of understanding people can feel calmer and more confident that they will be able to comprehend the rest of the information. Also, having two different examples with different levels and factors of uncertainty allows the coinciding information about COVID-19 much easier to comprehend.

I believe that the thoroughness and the many examples given when exploring what risk and uncertainty mean at a fundamental level are extremely helpful for non-quantitively-inclined readers. One of the main objectives of the essay is exactly giving a quantitative look at a problem many people view only qualitatively (will happen, may happen, won't happen). Considering there is no escape from the quantitative analysis, I believe the way the essay handles it is excellent, by explaining at a basic level what is risk and uncertainty, using several examples along the way, in a way that almost all readers paying attention and that understand the basics of fractions and percentages will be able to understand. I also believe that the connection back to the pandemic will be specially illuminating for readers who hadn't considered uncertainty in risk from a quantitative POV before, as it equips them to better think about the numbers and predictions that are being shown to them in the media, and reflect on what they really mean, and what would be a reasonable response.

Unequivocally, the passage in this essay on COVID statistics was eye-opening to read for someone "non-quantitatively-inclined" such as me. While the language of low-uncertainty and high-uncertainty experiences was highly interesting and illuminating, I can't imagine organically changing behaviour based on these discoveries. When the statistics are set out in such a way that they are in the essay though; in plain language, and particularly in comparative context; to the average annual death rate, to the death rate of the Spanish flu, etc., I feel like I'm being introduced to a coherent narrative, based in fact, which will tangibly impact my thoughts, opinions, and thus behaviours.

The content, then, I think is excellent! It sounds silly, but perhaps a good way to impart it other than in essay format would be through a TikTok. While a short, tiktok format videos starts off mostly being watched by the younger demographic that likes the app, I see the videos being shared now frequently on other platforms, such as Twitter and Facebook, which have a larger reach for broader swathes of the population!

I surely agree, @vincentli -- these are the draft images we ultimately didn't include, but clearly should!

I appreciated the real-world examples of the combinations of risk and uncertainty (whether high or low for each). They concretely demonstrated the difference between risk and uncertainty in a way accessible for all readers. Perhaps including a graphic with the example scenarios (e.g. 2x2 table of risk on horizontal axis and uncertainty on vertical axis, similar to a Prisoner’s Dilemma table) would be a quick way to reference the difference between risk and uncertainty. Overall, I thought the essay addressed an important topic, since risk and uncertainty are often used interchangeably in everyday conversation but have different meanings.

So for example, if we see the coronavirus is infecting people in the United States, the known known is that it affects older people more and people with certain preexisting conditions. We also might have some sparse data and some rudimentary understanding of the mechanisms that lead to the spread of the virus and the way it leads to hospitalizations and deaths. From this, we might calculate a risk (or the odds) of death and hospitalizations. However, this risk may have a large uncertainty because we know that we do not know a whole number of things like does it spread through inanimate objects or how capable is the government in withstanding public anger when initiating lockdowns or how many people will actually wear masks. This might give us an initial range of uncertainty but then we also have the unknown unknown items such as we have no idea if another natural disaster occurs or if another country invades another country or how and when will the virus mutate. These are all unknown unknowns that further exacerbate the uncertainty. If there is good reason to worry that tail-risks can have dramatic impact on the outcome of the pandemic on # of deaths, then we might model with fat tails to represent the unknown unknown and we might keep a large range to model the high uncertainty reflected in the known unknowns.

Similar to Daniel, I think the most illuminating part of this article for non-quantitatively-inclined readers is the discussion comparing low-uncertainty experiences and high-uncertainty experiences in our lives. This comparison of events that we can relate to, allows us as readers to imagine our decision-making in these situations. Thus, we are able to evaluate the way we actions we make in our lives, and the way in which uncertainty affects them.

Similarly to what Will has written, I think that the part of the article most illuminating for non-quantitatively inclined readers would be a very number-heavy paragraph. In the paragraph that begins with "What fraction of people....", Prof Goodman and Prof De Vivo clearly spell out the calculations for what the fraction of people having the experience being evaluated die from COVID-19 is. Because this solution is prefaced with step-by-step arithmetic, this part of the discussion will be most illuminating for non-quantitatively-inclined readers such as myself.

I think the most illuminating part of the discussion for non-quantitatively-inclined readers is the part discussing the difference between low-uncertainty experiences and extremely high-uncertainty experiences. This part illustrates a good idea of what uncertainty is without using numerical data. Furthermore, the comparison between hunting and russian roulette prior gives a good explanation of the difference between risk and uncertainty.

The part of the discussion that I think would be most illuminating to non-quantitative readers is this idea of understanding risk and uncertainty and how that relates to the news. As we know, the new channels are fighting for attention and in order to increase eyeballs they usually use misleading titles or data because the crazier it looks, the more likely they are to lure someone in. I think regardless of how quantitative someone is, they can always think about the uncertainty of these news article predictions and understand that no matter what there’s always a chance that these numbers are misleading. I think this could help people take a step back and stop immediatly forming life decisions or actions based on a quick glimpse of a news title or graph.

After reading Profs. Goodman and de Vivo's article titled "Uncertain Risks," I believe that paragraph six (beginning with "The fraction scientists seek") is the most effective section of the piece at developing understanding of risk and uncertainty for non-quantitatively inclined readers. Primarily, it conveys very clearly the different methods of quantitatively communicating risk and uncertainty. Although it may be obvious to some, parsing out the quantitative synonyms—like "1-in-a-hundred" and "1%"—and commonly misunderstand concepts—like "5% uncertainty"—is incredibly valuable.