A4

Documenting Uncertainty

A4.1 Introduction

The authors have assessed a wide range of information in the scientific literature and various technical reports to arrive at their Key Findings. Similar to the 2014 NCA1 and the Intergovernmental Panel on Climate Change’s Fifth Assessment Report,2 this assessment relies on two metrics to communicate the degree of certainty in Key Findings:

  • Confidence in the validity of a finding based on the type, amount, quality, strength, and consistency of evidence (such as mechanistic understanding, theory, data, models, and expert judgment); the skill, range, and consistency of model projections; and the degree of agreement within the body of literature.
  • Likelihood, or probability of an effect or impact occurring, is based on measures of uncertainty expressed probabilistically (in other words, based on statistical analysis of observations or model results or on the authors’ expert judgment).

Key sources of information used to develop these characterizations are referenced in the Supporting Evidence section found at the end of each chapter. The Supporting Evidence sections include “Traceable Accounts” for each Key Finding that 1) document the process and rationale the authors used in reaching the conclusions in their Key Finding, 2) provide additional information to readers about the quality of the information used, 3) allow traceability to resources and data, and 4) describe the level of likelihood and confidence in the Key Finding. Thus, the Traceable Accounts represent a synthesis of the chapter author team’s judgment of the validity of findings, as determined through evaluation of evidence and agreement in the scientific literature. The Traceable Accounts also identify areas where data are limited or emerging or where scientific uncertainty limits the authors’ ability to estimate future climate change impacts. Each Traceable Account includes 1) a description of the evidence base, 2) major uncertainties, and 3) an assessment of confidence based on evidence.


A4.2 Evaluation of Confidence in the Validity of a Finding

Assessments of confidence in the Key Findings are based on the expert judgment of the chapter authors. Authors provide supporting evidence for each of the chapter’s Key Findings in the Traceable Accounts. Confidence is expressed qualitatively and ranges from low confidence (inconclusive evidence or disagreement among experts) to very high confidence (strong evidence and high consensus) (see below). Confidence levels are reported even where confidence is low. Confidence should not be interpreted probabilistically, as it is distinct from statistical likelihood.

Likelihood and Confidence Level

Likelihood

Very Likely
≥9 in 10
Likely
≥2 in 3
As Likely as Not
≈ 1 in 2
Unlikely
≤ 1 in 3
Very Unlikely
≤1 in 10

Confidence Level

Very High Strong evidence (established theory, multiple sources, consistent results, well documented and accepted methods, etc.), high consensus
High Moderate evidence (several sources, some consistency, methods vary and/or documentation limited, etc.), medium consensus
Medium Suggestive evidence (a few sources, limited consistency, models incomplete, methods emerging, etc.), competing schools of thought
Low Inconclusive evidence (limited sources, extrapolations, inconsistent findings, poor documentation and/or methods not tested, etc.), disagreement or lack of opinions among experts

A4.3 Evaluation of Likelihood of Risk

For the purposes of this assessment, likelihood is the chance of occurrence of an effect or impact based on measures of uncertainty expressed probabilistically (in other words, based on statistical analysis of observations or model results or on expert judgment). Authors came to a consensus using expert judgment, based on the synthesis of the literature assessed, to arrive at an estimation of the likelihood that a particular impact will occur within the range of possible outcomes. Where it is considered justified to report the likelihood of particular impacts within the range of possible outcomes, this report takes a plain-language approach to expressing the expert judgment of the chapter team, based on the best available evidence. For example, an outcome termed “likely” has at least a 66% chance of occurring; an outcome termed “very likely,” at least a 90% chance (see Figure 1).


A4.4 Uncertainty Language in Key Findings

All Key Findings include a description of confidence. Where it is considered scientifically justified to report the likelihood of particular impacts within the range of possible outcomes, Key Findings also include a likelihood designation. Where possible, levels of confidence and likelihood are provided for different steps along the exposure pathway to enable separate reporting of levels of uncertainty in understanding climate impacts, changes in exposure, the role of moderating or exacerbating factors, and observed or projected health outcomes.

Confidence and likelihood levels are based on the expert assessment and consensus of the chapter author teams. These teams determined the appropriate level of confidence or likelihood by assessing the available literature, determining the quality and quantity of available evidence, and evaluating the level of agreement across different studies. Often, the underlying studies provided their own estimates of uncertainty and confidence intervals. When available, these confidence intervals were assessed by the chapter authors in making their own expert judgments. For specific descriptions of the process by which each chapter author team came to consensus on the Key Findings and the assessment of confidence and likelihood, see the Traceable Accounts in the Supporting Evidence section of each chapter.

References

  1. IPCC, 2014: Climate Change 2014: Impacts, Adaptation, and Vulnerability. Part A: Global and Sectoral Aspects. Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. 1132 pp., Cambridge University Press, Cambridge, UK and New York, NY. URL | Detail
  2. Melillo, J. M., T. (T. C. ) Richmond, and G. W. Yohe, eds., 2014: Climate Change Impacts in the United States: The Third National Climate Assessment. U.S. Global Change Research Program, 841 pp. doi:10.7930/J0Z31WJ2 | Detail

Likelihood

Very Likely
≥9 in 10
Likely
≥2 in 3
As Likely as Not
≈ 1 in 2
Unlikely
≤ 1 in 3
Very Unikely
≤1 in 10

Confidence Level

Very High Strong evidence (established theory, multiple sources, consistent results, well documented and accepted methods, etc.), high consensus
High Moderate evidence (several sources, some consistency, methods vary and/or documentation limited, etc.), medium consensus
Medium Suggestive evidence (a few sources, limited consistency, models incomplete, methods emerging, etc.), competing schools of thought
Low Inconclusive evidence (limited sources, extrapolations, inconsistent findings, poor documentation and/or methods not tested, etc.), disagreement or lack of opinions among experts
 

Documenting Uncertainty: This assessment relies on two metrics to communicate the degree of certainty in Key Findings. See Appendix 4: Documenting Uncertainty for more on assessments of likelihood and confidence.