Weiye Loh's Library tagged Projection → View Popular, Search in Google

Jan

2012

Roger Pielke Jr.'s Blog: US Emissions Projections Compared to Reduction Targets

It should be fairly obvious that under the assumptions of the EIA (such as positive economic growth) that the emissions reduction targets are not going to be met. Given President Obama's renewed commitment to an "all of the above" strategy for energy production in the United States, is it finally time to dismiss the charade of emissions reductions targets and adopt a different approach?

rogerpielkejr.blogspot.com/...s-projections-compared-to.html

- Preview

Carbon Emission Projection Economics

The graph at the top of this post shows the U.S. government's emissions projections (black line) and the emissions reduction targets (red, blue and green). In case you were wondering how big the misses are with respect to the targets in some sort of intuitive way, I've provided a measure of the magnitude of the shortfall (using the same methods described in depth in The Climate Fix) in terms of the number of coal power plants that would have to be replaced with nuclear power plants to meet the targets. (If you'd like to replace gas power plants, the numbers are about 40% more, due to the lower carbon intensity of gas generation. If you'd like to use wind turbines or solar power, well, get out a big calculator;-).

Aug

2011

Roger Pielke Jr.'s Blog: How Many Findings of the IPCC AR4 WG I are Incorrect? Answer: 28%

Jonassen, R. and R. Pielke, Jr., 2011. Improving conveyance of uncertainties in the findings of the IPCC, Climatic Change, 9 August, 0165-0009:1-9, http://dx.doi.org/10.1007/s10584-011-0185-7.

Abstract Authors of the Intergovernmental Panel on Climate Change (IPCC) Fourth Assessment Report (AR4) received guidance on reporting understanding, certainty and/or confidence in findings using a common language, to better communicate with decision makers. However, a review of the IPCC conducted by the InterAcademy Council (2010) found that "the guidance was not consistently followed in AR4, leading to unnecessary errors . . . the guidance was often applied to statements that are so vague they cannot be falsified. In these cases the impression was often left, quite incorrectly, that a substantive finding was being presented." Our comprehensive and quantitative analysis of findings and associated uncertainty in the AR4 supports the IAC findings and suggests opportunities for improvement in future assessments.

rogerpielkejr.blogspot.com/...ings-of-ipcc-ar4-wg-i-are.html

- Preview

Climate Change Climate Science Policy Probability Prediction Projection IPCC

If we confine our attention to those findings that refer to the future, one can ask how many IPCC findings can be expected to become verified ultimately as being accurate? For example, if we consider findings that refer to future events with likelihood in the ‘likely’ class (i.e., >66% likelihood) then if these judgments are well calibrated then it would be appropriate to conclude that as many as a third can be expected to not occur. More generally, of the 360 findings reported in the full text of WG1 across all likelihood categories and presented with associated measures of likelihood (i.e., those summarized in Table 2 below), then based on the judgments of likelihood associated with each statement we should logically expect that about 100 of these findings (~28%) will at some point be overturned.
ur paper concludes:

Although the IPCC has made enormous contributions and set an important example for global assessment of a vexing problem of immense ramifications, there remain clear opportunities for improvement in documenting findings and specifying uncertainties. We recommend more care in the definition and determination of uncertainty, more clarity in identifying and presenting findings and a more systematic approach in the entire process, especially from assessment to assessment. We also suggest an independent, dedicated group to monitor the process, evaluate findings as they are presented and track their fate. This would include tracking the relationship of findings and attendant uncertainties that pass up the hierarchy of documents within AR5. Strict rules for expressing uncertainty in findings that are derived from (possibly multiple) other findings are needed (see, e.g., the second example in the Supplementary Material).

It is not the purpose of this note to discuss other, related scientific assessments of climate change knowledge; but, we do note that our preliminary analysis of the U.S. Global Change Research Program Synthesis and Assessment Products suggests a far less systematic application of the guidance supplied to authors of those documents and far less consistent application of the defined terms. We believe that the concerns we have expressed here, and the resulting recommendations, apply more broadly than the IPCC process.
12 more annotation(s)...
- Your headline is of course utter nonsense.
  
  If I say (correctly) that a single roll of a fair die is "likely" (83%) to turn up 1-5, then throwing a 6 does not make my statement "incorrect". If I repeat the statement (and experiment) 100 times, then 16% of my statements will not have been incorrect either.
- -1-James Annan
  
  Thanks for the comment. If the IPCC statements were in reference to die rolls you'd be correct. But they are not.
  
  A finding of the IPCC such as the following will either be right (it will verify) or it will be wrong (it will not), the IPCC handicaps it at 90% certain:
  
  "It is very likely that hot extremes, heat waves and heavy precipitation events will continue to become more frequent."
  
  There is a 10% chance, according to the IPCC, that the statement will not verify.
  
  There are not 100 die rolls here, just one future.
- Why does your analysis not apply to a roll of a die? If it helps you, assume that after the first throw, a magpie swoops down and steals it. The throw is not repeatable. Would you claim I had been incorrect if I had thrown a 6?
  
  The statements you analyse are still probabilistic statements irrespective of repeatability, and as such can be neither absolutely verified nor refuted in isolation by the future. If 10% of their very likely statements turn out to not occur, this would simply mean their assessment was well calibrated.
  
  Therefore your usage of "incorrect" (which interesting does not appear in the paper, only the blog post) is nonsensical.
  
  I think you know this and are playing at rabble-rousing. My eyebrows are raised, but only a little, at this behaviour.
- -4-James Annan
  
  If I ask you to predict the next roll of a die and you predict that it will be in the range 1-5, an it turns up 6, then you would be wrong.
  
  You could hem and haw about 100 rolls and such, but I would explain that I was not asking for a discourse on probabilities but a simple prediction of the next roll.
- we may both wish that the IPCC was more sophisticated in its discussion of probabilities and futures, and that is a subject of our paper.
  
  Operational weather forecasters will no doubt understand this situation quite well. To suggest that there is no such thing as being incorrect, probably does not help the credibility of climate science.
- But if, rather than making a definitive prediction that it *will* be in the range 1-5, I merely state that this is "likely" to occur, will you still say I was wrong if it comes up a 6?
  
  It seems to me that the IPCC was rather more sophisticated in its usage of probability, than you are in understanding it.
- The probabilities themselves are basically untestable, as far as whether they reflect reality, vis a vis the "real" PDF, unless there is a 1 or a 0 in there somewhere.
  
  The typical probabilistic statement is probably too vague to really be able to pin it down and decide what it really means. As always, the real sticking point comes down to the assumptions behind the probabilities.
  
  Of course, there's also the fact that (from what I've seen) many of the probabilities are about parameters, not actual data, so the predictions probably don't mean what most people will think they mean.
- with respect to the IPCC PDFs are not known, and the report is written about the climate future, of which we will have only one.
  
  Following your logic the IPCC can never be evaluated empirically about anything expressed probabilistically. This is silly. Probabilistic statements are made to add information to decision making, not to subtract from accountability.
  
  If a weather forecaster offers an 80% chance of rain the next day and does so for 360 days, then I can evaluate that forecast by converting it into a deterministic prediction that should verify 80% of the time, if it is well calibrated. Similarly, if the IPCC offers up 360 statements expressed with various certainties, then (if its judgments are well-calibrated) we can expect that the scientific community will collectively judge 28% of those statements to be incorrect in the future.
- Actually, the IPCC offers PROJECTIONS, not predictions. Projections are the result of running models based on a certain scenario, or "storyline", up to a certain point in the future: if the storyline turns up to be correct, AND the model actually embodies the climatic implications of the storyline, then the projection would agree with observation. Logically this does not imply that the storyline is correct, and that's the reason why several such storylines (scenarios) are used. To the uncertain storyline one has to add the uncertainties in the models themselves: they could be structurally wrong, or some parameter of the model may have been wrongly quantified (e.g. climate sensitivity).
- Probabilities refer always to collections of events, not to specific events. They make statements about the relative frequency of an outcome among the various possible outcomes, over a large number of observations. They can be falsified by collection of events, not by individual events. The 1/6 probability of getting a six in one throw of a die is not falsified by the next throw not showing a six, or even by several throws failing to show a six. Twenty or fifty (or more) throws without a six are definitely possible. It will not be confirmed by getting a six in the next throw. But a large number of throws (by the Law of Large Numbers) will show 1/6 of the cases were sixes. This hardly applies to climate projections, because only ONE future would be actually observable, and that will happen in the future and is thus no help to evaluate the projections in the present.
- Most of the "likely" or "very likely" assessments in the IPCC reports are based on ensembles of models, each with its particular quirks and peculiar quantification of parameters. If 66% of those model runs show a certain result (e.g. average temperature rise above 2°) this is not translatable into 66% probability of such warming to occur: other scenarios and models, or other quantifications of the models, would yield different results.
  
  One must also distinguish probability and strength of belief, though in some context (especially those using Bayesian approaches) the two are often not distinguished. If a die fails to show a six after fifty throws, our belief in the fairness of the die starts to falter, but perhaps the die is fair after all, and we are also facing a long succession of no-sixes which is entirely compatible with the die being fair.
  
  The likelihood attributed to future events in IPCC reports are more akin to strength of belief (among IPCC authors) than to probabilities.
- I thought the exchange between Annan and Pielke Jr was amusing, because Pielke’s headline – 28% of the WG1 findings are wrong – was tongue in cheek. He doesn’t really believe that (as his text itself reveals), but was just being provocative. In a broader sense, the argument reminds us that “probability” is not an inherent property of a system but a measurement of our ignorance about the system. In the case of climate, as we learn more about particular phenomena, the probability of their occurrence will change, even though our learning process hasn’t had any impact on how the natural world behaves. A future event that is 72% probable now may become 99% probable in the future (or 1% probable) even if we have done nothing to change its behavior.

Mar

2011

Roger Pielke Jr.'s Blog: It Is Always the Media's Fault

rogerpielkejr.blogspot.com/...it-is-always-medias-fault.html

- Preview

Media Science Communication Climate Science Oil Uncertainty Projection Forecast

Last summer NCAR issued a dramatic press release announcing that oil from the Gulf spill would soon be appearing on the beaches of the Atlantic ocean. I discussed it here.

Here are the first four paragraphs of that press release:

BOULDER—A detailed computer modeling study released today indicates that oil from the massive spill in the Gulf of Mexico might soon extend along thousands of miles of the Atlantic coast and open ocean as early as this summer. The modeling results are captured in a series of dramatic animations produced by the National Center for Atmospheric Research (NCAR) and collaborators.

he research was supported in part by the National Science Foundation, NCAR’s sponsor. The results were reviewed by scientists at NCAR and elsewhere, although not yet submitted for peer-review publication.

“I’ve had a lot of people ask me, ‘Will the oil reach Florida?’” says NCAR scientist Synte Peacock, who worked on the study. “Actually, our best knowledge says the scope of this environmental disaster is likely to reach far beyond Florida, with impacts that have yet to be understood.”

The computer simulations indicate that, once the oil in the uppermost ocean has become entrained in the Gulf of Mexico’s fast-moving Loop Current, it is likely to reach Florida's Atlantic coast within weeks. It can then move north as far as about Cape Hatteras, North Carolina, with the Gulf Stream, before turning east. Whether the oil will be a thin film on the surface or mostly subsurface due to mixing in the uppermost region of the ocean is not known.
A few weeks ago NCAR's David Hosansky who presumably wrote that press release, asks whether NCAR got it wrong. His answer? No, not really:

During last year’s crisis involving the massive release of oil into the Gulf of Mexico, NCAR issued a much-watched animation projecting that the oil could reach the Atlantic Ocean. But detectable amounts of oil never made it to the Atlantic, at least not in an easily visible form on the ocean surface. Not surprisingly, we’ve heard from a few people asking whether NCAR got it wrong.

These events serve as a healthy reminder of a couple of things:

*the difference between a projection and an actual forecast
*the challenges of making short-term projections of natural processes that can act chaotically, such as ocean currents
4 more annotation(s)...
- What then went wrong?
  
  First, the projection. Scientists from NCAR, the Department of Energy’s Los Alamos National Laboratory, and IFM-GEOMAR in Germany did not make a forecast of where the oil would go. Instead, they issued a projection. While there’s not always a clear distinction between the two, forecasts generally look only days or hours into the future and are built mostly on known elements (such as the current amount of humidity in the atmosphere). Projections tend to look further into the future and deal with a higher number of uncertainties (such as the rate at which oil degrades in open waters and the often chaotic movements of ocean currents).
  
  Aware of the uncertainties, the scientific team projected the likely path of the spill with a computer model of a liquid dye. They used dye rather than actual oil, which undergoes bacterial breakdown, because a reliable method to simulate that breakdown was not available. As it turned out, the oil in the Gulf broke down quickly due to exceptionally strong bacterial action and, to some extent, the use of chemical dispersants.
- Second, the challenges of short-term behavior. The Gulf's Loop Current acts as a conveyor belt, moving from the Yucatan through the Florida Straits into the Atlantic. Usually, the current curves northward near the Louisiana and Mississippi coasts—a configuration that would have put it on track to pick up the oil and transport it into open ocean. However, the current’s short-term movements over a few weeks or even months are chaotic and impossible to predict. Sometimes small eddies, or mini-currents, peel off, shifting the position and strength of the main current.
  
  To determine the threat to the Atlantic, the research team studied averages of the Loop Current’s past behavior in order to simulate its likely course after the spill and ran several dozen computer simulations under various scenarios. Fortunately for the East Coast, the Loop Current did not behave in its usual fashion but instead remained farther south than usual, which kept it far from the Louisiana and Mississippi coast during the crucial few months before the oil degraded and/or was dispersed with chemical treatments.
- The Loop Current typically goes into a southern configuration about every 6 to 19 months, although it rarely remains there for very long. NCAR scientist Synte Peacock, who worked on the projection, explains that part of the reason the current is unpredictable is “no two cycles of the Loop Current are ever exactly the same." She adds that the cycles are influenced by such variables as how large the eddy is, where the current detaches and moves south, and how long it takes for the current to reform.
  
  Computer models can simulate the currents realistically, she adds. But they cannot predict when the currents will change over to a new cycle.
  
  The scientists were careful to explain that their simulations were a suite of possible trajectories demonstrating what was likely to happen, but not a definitive forecast of what would happen. They reiterated that point in a peer-reviewed study on the simulations that appeared last August in Environmental Research Letters.
- So who was at fault? According to Hosansky it was those dummies in the media:
  
  These caveats, however, got lost in much of the resulting media coverage.
  Another perspective is that having some of these caveats in the press release might have been a good idea.