Sunday 31 July 2016

Predicting 2016 from June - Satellite Data

Disclaimer

All the temperature sets are in for June 2016, so it's time to update my predictions. But first I want to emphasize that I don't claim to agree with these forecasts. My main intention was to disagree with those claiming back in April that there was 99% certainty that 2016 would set a record. My assumption has always been that the earlier predictions were overstated, and that it would become clear one way or another over the coming months. But at this midway point I think things are even less clear, and may be exaggerating the likelihood of a record even more.

I also want to point out that whilst speculating on which years will be records is interesting, it doesn't tell us much about the trend in global warming. Annual temperatures are fairly arbitrary. A single record does not prove global warming is happening, and a lack of records certainly does not prove warming is not happening.

Roy Spencer of UAH has been talking about the likelihood of UAH beta 6 setting a record as well. In June he was predicting that 2016 will likely be record in the satellite data., but after the June figures showed a large drop he's now suggesting it's unlikely to set a record . So I want to look in more detail at UAH beta 6, which is the least typical of all the data sets. Spoiler - I also think it's unlikely to set a new record this year.

Updates for June

I'm monitoring 7 data sets; the three main terrestrial sets - GISTEMP, NOAA and HadCRUT, and 4 versions of the satellite data - UAH 5.6, UAH beta 6, RSS 3.3, and RSS 4.0. RSS has not so far released version 4 for the lower troposphere so I'm using their product for the total troposphere (TTT), which is the closest to TLT. Here's how the years been progressing for each set (all anomalies based on the period 1981 - 2010).

Most of these data sets showed a drop in June. The exceptions being NOAA and HadCRUT that both showed a small rise.

Here are the graphs showing how the projected year end total has changed throughout the year.

This graph shows the expected annual temperature compared with the previous record year (1998 for satellite data, 2015 for terrestrial data). All of the data sets show a drop in the forecast (compared with the forecast from May), except for HadCRUT which has a small rise. How has this effected the probability of a record year?

Surprisingly there hasn't been much change, with the noticeable exception of UAH beta 6, which drops from 88% to 80%. All others show 90% - 100% certainty. RSS 3.3 and HadCRUT swapped places, with RSS dropping from 94% to 92%, and HadCRUT rising from 92% to 95%. NOAA remains at 98% and the others, GISTEMP and the other satellite versions are greater than 99%.

Summary of Forecasts
Set Probability Margin
GISTEMP 1.00 0.16
HadCRUT 0.95 0.10
NOAA 0.98 0.11
RSS 3.3 0.92 0.09
RSS 4.0 1.00 0.20
UAH 5.6 1.00 0.21
UAH beta 6 0.80 0.05
How The Calculations Were Made

In order to get some idea of where these forecasts come from I'll go through the process using UAH beta 6 as an example. For context here's the annual observations along with the forecast (in red) for 2016. The vertical line indicates the 95% interval.

The forecast is to be quite close to the 1998 record, and much warmer than any other year.

The forecast is based on the correlation between the annual temperature and the first 6 months for each year.

This suggests there has been a strong linear correlation between the temperature at the start of the year and the final temperature. There are a few reasons why this correlation shouldn't be a surprise

  • The annual figure includes the initial values. In other words, by this point half of the annual figure has already been determined.
  • The temperatures for both the start of the year and the whole year are correlated with the underlying trend. That is, because temperatures have increased over time, and years that starts warm are likely to be from the warmer period, and so are more likely to see a warmer end to the year.
  • There is some correlation between the start and end of the year even allowing for the underlying trend. That is years that start unusually hot for their time are more likely to remain unusually hot.

The expected temperature can be read straight from that graph. The start of 2016 has been a little warmer than the start of 1998, and 1998 was quite close but a little below the predicted value.

One note of caution is that a fair proportion of the trend line is only supported by two years, 2010 and 1998. On the other hand, it does show that both the strong El Niño years finished up close to their predicted values.

The trend line gives us the expected temperature for the year. This is the temperature we'd expect to get on average, if we had a large number of years all starting with the same temperature. In order to estimate a probability of beating any specific temperature we need to generate a distribution for the range of specific values. This is obtained by calculating the prediction interval (by which I mean I get R to calculate it). I then hack the figures a bit to get the correct scale. The distribution looks like this.

The dotted line shows the 1998 record anomaly, and the shaded are shows where 2016 has to be to beat it. This translates into an 80% probability of beating the 1998 record.

How Plausible are These Forecasts?

Taking another approach (that used by Roy Spencer) we can look at what actually needs to happen in the second half of 2016 to beat the record. For UAH beta 6 the second half of 2016 has to average more than 0.348 °C (using UAH's 1981 - 2010 base period). This compares to the first 6 months of 2016 which was 0.617 °C, and the second half of 1998, which was 0.373 °C.

For 2016 not to be a record would require the biggest difference between the first and second halves of the year in the UAH record. The biggest difference was in 1998 when temperatures dropped by 0.239 °C. By contrast 2016 will need to drop by 0.269 °C - so either way 2016 will set some sort of record!

Here's what 2016's monthly values look like compared with 1998, with the dotted line showing the average temperature needed for the rest of the year to break the record.

Although 2016 started off considerably warmer than 1998, the last two months have shown a much quicker drop than anything seen in 1998. Temperatures will actually need to increase from the June figure to beat the record, but if 2016 reflects the pattern of 1998 temperatures should continue dropping. If this happens there would be no chance of 2016 beating 1998.

It seems likely that temperatures will continue to drop, as they did in 1998, but it's not certain as 2016 is not behaving in the same way as 1998. Overall, based on intuition rather than statistical inference, I suspect the odds on UAH beta 6 setting a record are low - say around 10%.

So why would the statistical analysis be so wrong? In some ways I don't think it is actually wrong, it's just that it predicts the general case. Saying there is an 80% chance of record means that if we had a large number of different years, all starting with the same temperature, 80% would be warmer than 1998, but it doesn't predict which specific years will be in that 80%. You can always bring more knowledge about what a specific year is doing that will suggest if this particular year will be in that 80% or not.

I have tried other statistical approaches, and want to introduce at some point one that gives a better overall performance. This combines the underlying trend with the start of the year, but I'm reluctant to use it at this stage as it actually increases the probability of a record, going in completely the wrong direction to what I expect to happen.

A more accurate statistical method would probably require more advanced methods, such as deep learning, which is outside my experience.

Additional Comments

Whilst it is interesting that despite the strong start to the year UAH beta 6 will probably not set record this year, that tells us little about the state of global warming. For a start we are only talking about one data set, and one that is still unofficial and unpublished. Whilst I'd reduce the odds on all the other data sets breaking a record, they all have more chance than UAH beta 6. My estimates for the other data sets would be that RSS 3.3 has around a 50% chance of setting a record, and the other sets all have 75% or more. (These estimates are as above just my gut feeling.)

UAH beta 6 is dominated by the exceptional value it gives for 1998. 2016 is still likely to be close to the 1998 value and 2016 is very likely to be at least the 2nd warmest year in their history, and is almost certain to be warmer than 2015.

What makes 1998 so unusual is that it came out of much cooler conditions. The two years preceding, and the two years following 1998 were all close to the 1981 - 2010 average. They were warm by 20th century standards, but cold compared to the 21st century. By contrast 2016 follows two very warm years. Possibly one reason this El Niño is dropping more quickly than 1998 is that it has already produced more heat over the previous year. This graph compares monthly values for the three years leading up to 1998 and 2016.

One consequence of this is that the two years 2015 and 2016 are almost certain to be the warmest 2 consecutive years on record, and the three years from 2014 - 2016 will be the warmest 3 consecutive years on record.

The Other Satellite Data Sets

For completeness, here's what the other versions need to do to beat their respective records..

All of them can afford to drop some more and still beat 1998.

Update - 1st August

No sooner had I written the above, when Roy Spencer announces that the UAH beta 6 value for July 2016 showed a slight increase on the June value - up to 0.39 °C.

This hasn't had too much of an impact on my forecasts. The statistical method shows a slight reduction in the probability of this data set breaking a record, down to 78%. I'd still agree with Spencer that UAH will beat the record this year, but the fact that the temperatures went up must increase the odds slightly, if only because it suggests more uncertainty in the temperatures.

Friday 15 July 2016

Referendum 3 - On the Brink of Brexit?

Yet another post about Brexit, in which I continue to whine about the unfairness of democracy, and swing between despair and cautious optimism.

So, we now have a new PM who got the job essentially by default, despite being the only one who was campaigning for the UK to remain in the EU. As you may have noticed, I'm one of the 48% who want to remain in the EU, so you might think there is some advantage to having a PM who claims to have wanted to remain, and who has said leaving would not be in the national interest.

But I fear the reality of the political situation is that it would have been easier for a Leaver to back out than for a Remainer. A Leaver, assuming they didn't really want to leave, could announce that after a lengthy consultation process they'd regretfully come to the conclusion that whilst they still had ideological objections to the EU, the current circumstances meant that the decision should be delayed until they could get a better deal, etc, etc.

If Theresa May wanted to do that, it will be painted as suspicious, that she never really wanted to leave, and had only said that Brexit means Brexit to win the leadership contest. Moreover, the fact that she's now created the post of Secretary of State for Exiting the European Union, might suggest she is not keeping an open mind and the question has already been answered.

Yet, and this is me clutching at whatever tiny straw a can, I do wonder if this might be a crafty maneuver to seem to be appeasing the quitters, whilst giving them enough rope. If in the next few months David Davis, one of the staunchest Euro-skeptics, cannot come up with a realistic plan, it could be just the excuse needed to delay any decision for foreseeable future, or reach some sort of compromise.

For some context here's a detailed article looking at the challenges the Brexit commissioner will face. It suggests the best (that is least worse) solution would be to delay things until 2020 using an interim agreement.

An even more optimistic sign is that, as I'm writing this, it's being reported that May has had a constructive meeting with Scotland's First Minister First Minister Nicola Sturgeon, and is saying Article 50 won't be triggered until she has a UK approach. This really begins to seem that she is trying to put the pressure on Davis - not only does he have to find a plan for leaving the EU, but it has to satisfy Scotland and Northern Ireland.

Thursday 7 July 2016

Referendum 2 - The Aftermath

Who is Slumkey? whispered Mr. Tupman.

I don't know, replied Mr. Pickwick in the same tone, Hush. Don't ask any questions. Its always best on these occasions to do what the mob do.

But suppose there are two mobs? suggested Mr. Snodgrass.

Shout with the largest, replied Mr. Pickwick.

The Pickwick Papers - Charles Dickens

Two weeks since the shambles of the the EU referendum, and I've been wondering if it's worth posting my options. I made it clear what I thought about the referendum in my previous post. I also made a few comments on various blogs, especially HotWhopper here, and here.

But a few things I need to get off chest. This post isn't going to be about why I think leaving the EU will be a bad decision, but on who's responsible for this mess.

The Story So Far

Here's a brief and partial summary of where we stand as of today - Thursday 7th July.

  • The referendum took place on the 23rd June - and was a narrow defeat for the UK, with 52% of voters voting to leave the EU, 48% to remain.
  • Nigel Farage gave a victory speech, even though he wasn't part of the official Leave team, somewhat extending the brief to announce his desire to destroy the EU entirely and return to a Europe of independent nation states. (Because that worked so well in the past.)
  • Early in the morning David Cameron announced his resignation and that he would leave it to his successor to trigger Article 50. Everyone who had been campaigning for Leave suddenly started talking about the need not to be hasty, and how important it was to think through our options.
  • Monday 27th June, Boris Johnson the main Leave campaigner, and widely seen as the inevitable next PM, spells out his vision the post EU UK. He does this in his weekly Daily Telegraph article, in which he stresses nothing much will change as a result of our leaving the EU, that we'll be more integrated with them, we'll still be able to work in the EU, but somehow we'll be controlling out borders. Quite how he can guarantee any of this when negotiations haven't even started is another question.
  • The next day it is suggested in some quarters that Johnson didn't really mean what he said, and was tired when he wrote the article.
  • Thursday 30th June - with the deadline for Conservative ministers to put their names forward at midday. First thing in the morning Gove, having previously saying he'd support Johnson's bid, stabs Boris in the back, and announces he'll be standing. Minutes before the deadline Johnson announces he will not be standing.
  • Monday 4th July, Farage resigned from UKIP. Johnson wrote another manifesto in his weekly Telegraph column suddenly realizing it would be a good idea to have some sort of plan for Brexit.
  • Today, Gove fails to make the final shortlist for Tory leadership, so the members will be choosing between Theresa May and Andrea Leadsom.

So we will have to wait till September to find out who will be leading the UK into this brave new future. But whoever it is, it won't be anyone who was leading the leave campaign. It won't be anyone who can be held to account for failing on the promises, and if May is elected it will be someone who was claiming to believe that it was not in the national interest to leave the EU (albeit someone who wants to abandon the European Court of Human Rights).

It seems no one had a plan for leaving the EU, not even those who campaigned for it. The whole thing has really been about internal conservative party politics, and the people will not get a say in what happens next. Yet for some reason everyone keeps insisting that whatever the cost to the UK, to Europe to the world, we must go through with the breakup. I really don't get why, and hope a more considered approach will be taken after the contest is over.

Does Brexit Mean Brexit?

The argument that the will of the people must be respected is flimsy and goes against the idea of parliamentary democracy. I can see three reasons to argue that the result of the referendum must be respected, without even a follow referendum: legal, pragmatic and moral. But none of them make sense to me.

The question of a legal obligations is just wrong. This was never a legally binding referendum; and if it had been the details would have needed to be much more rigorous. There should have been a requirement for a significant majority, there should have been a concrete proposal for what leaving would mean, and there should have been scrutiny of the claims maid during the campaign.

The pragmatic argument would be that the electorate would punish the party that failed to deliver on leaving. This would require making a lot of assumptions about why people voted to leave, especially given the narrowness of the vote. How many who voted leave felt strongly enough about it to affect their decision at a general election? How many will have changed their mind by the next election? How many would never vote Conservative? How many would vote Conservative even if they reneged on leaving? And then reverse the question with the 48% who voted to remain.

I expect a shrewd PM who believed remaining would be in the national interest, would be able to frame the decision to go against the referendum, would be able to frame the action in a way that would not damage their chances in 2020. In any event the alternative would be taken the flack for all the damage caused by leaving, and being blamed for putting short term electoral interest above the country's good.

The moral argument could be that a promise was made and it would be immoral to break it. But politicians break promises all the time, and it could easily be argued that breaking a promise was the morally correct action, if keeping a promise was going to lead to worse consequences. In any event, the promises were made by people who are no longer in charge, and other promises, such as the 350 million a week to be spent on the NHS have already been broken.

Friday 1 July 2016

Prediciting 2016 - May update.

Now that we have all the main temperature sets for May 2016, it's time to update my forecasts for the year. See the previous post for details - I'm not making any claims about the actual probability of new records for 2016, I'm just performing a simple linear extrapolation from previous years.

May across the various data sets was sharply down on the high point of the El Niño, and some of the alarmists on places such as Watts Up With , this will inevitably lead to a new ice-age. This has had some impact on the projections for the final temperature of 2016. All data sets show a drop in the forecast of around 0.01 - 0.03 °C, compared with the forecast from April.

Here's the graph showing this for each data set I'm tracking. In case it isn't clear the y-axis is showing the forecast for 2016 compared with the previous record for that data set. In the case of ground based data sets, (GISS, NOAA and HadCRUT) this shows how much warmer 2016 is projected to be over 2015, in the case of the satellite data the comparison is with 1998. The x-axis shows the forecast that would have been made at each month of the year - so the left most edge is showing the forecasts made after the January temperatures are known, the rightmost side are the forecasts based on temperature from January to May.

However this has had practically no effect on the calculated probability of any data set breaking the record. In fact to the nearest percentage point, only RSS 3.3 has changed, from 94% to 93%.

I should stress, that I don't believe these probabilities, the real probability should be reduced to take into account unknowns, and the unusual start to the year. All the above is just for fun, and I take no responsibility for gambling losses.

Footnote

As I write this, Roy Spencer has released the June figures for UAH beta 6, and this shows another sharp drop. I'll look at this in more detail when more results are known, but this will have a bigger effect on the forecast. Including the result for June pushes the UAH beta 6 probability of 2016 being a record down to around 80%, with the expected value now only 0.05 °C above 1998.