Endgame

What can we expect now from the next general election?

Let’s take stock. Rishi Sunak has been Prime Minister for a year. Where are we heading?

Where are we now?

We can’t work out where we’re going without first understanding where we are now. The standard approach to this is to start by looking at the polling. I don’t intend to start there.

Polling is never very reliable — there are all kinds of variables that make it difficult to do accurately — but at present I’m particularly uncomfortable relying on uncorroborated polling for making predictions. Please note, this isn’t an attack on pollsters, who do a stupendously good job of a stupendously difficult job, but an attack on those who take what can only be fairly weak evidence and place far too much reliance on it.

There’s a whole separate post in why I am so uncomfortable about polling at present — I had basically written it already before I decided that it unbalanced this one — but the short version is that when we compare opinion polls with actual votes cast it looks as though Reform is seriously overstated in almost all polls (probably because it has too many extremely online supporters who confound polling samples) and the Greens also seem to be seriously overstated in most polls (possibly because too many people opt to make answering an opinion poll a proxy for their enthusiasm rather than their eventual choice).

All this means that roughly 10% of current polling vote shares in most polls looks as though it may be misallocated, and that’s before we get into the question of whether the problems reflected in recording the vote share of those parties are relevant to the respondents for other parties. There are visible problems with the polling results we’re seeing. Using them without corroboration is skating on particularly thin ice.

With the size of Labour’s lead at present, it probably doesn’t seem too important if polling isn’t very accurate so long as it is in the right ballpark. It’s worth noting, however, that the polling miss in 1997 was particularly large. That wasn’t particularly remarked on at the time because Labour won so well anyway. Still, it might matter this time. Labour start from much further behind and have a much steeper mountain to climb.

So instead of looking at the polls, let’s start from actual votes cast and see what we think is going on from that angle. We can then cross-refer it to current polling.

What can we tell from recent by-elections?

By-elections first. The first three by-elections of the 12 months were all in Labour-held seats with the Conservatives second. All swung to Labour by 10% or so. That’s respectable, but well below the swing that the polling headline numbers would suggest.

We had a gap of nearly six months, and then the next three were all held on the same day in July. All three were in Conservative-held seats. Somerton & Frome swung away from the Conservatives by a massive 29%, with the Lib Dems taking it with ease. Selby & Ainsty swung away from the Conservatives by just under 24%, to give Labour a memorable victory. The Conservatives managed, however, to hold Uxbridge & South Ruislip, suffering an adverse swing of only 6.7%.

We then had an interlude over the summer, broken by Labour taking Rutherglen & Hamilton West from the SNP on a swing of 20%. A week later, Labour took both Tamworth and Mid Bedfordshire on swings of 24% and 20% respectively.

Some unwonted humility

I have done pretty well with predictions this year, but by-elections are a glaring exception. I was initially expecting Labour to take Uxbridge effortlessly and they flunked it. A week in advance, I predicted that the Conservatives would hold both Tamworth and Mid Bedfordshire: wrong on both counts.

Still, one good thing about making mistakes is that you can learn from them. And there are lessons for me to learn. I underestimated how effectively the Conservatives could use the ULEZ expansion in Uxbridge to shift from defence to attack. I thought I saw some signs in the polling before Tamworth and Mid Bedfordshire that the Conservatives were starting to lift off the bottom: they weren’t.

How should we interpret these by-elections?

The general take from recent by-elections is that they show that current polling is more or less accurate. That begs the question: which polls? At one end of the spectrum we have More In Common with a Labour lead of 12%. At the other end, we have People Polling with a Labour lead of 28%. The average of the last poll leads of the 11 different pollsters that have reported in the last two weeks gives a Labour lead of 17% or so. As you can see, that covers a very big span.

David Herdson provided a convenient rule of thumb about by-election swings: “My rule of thumb is that by-elections swing by about 1.4 times the national polling”. I very much like this rule of thumb, which allows for differential turnout at by-elections. It also gives us reference points for testing the evidence we have in front of us.

Taking a simple average, the Conservative to Labour swing in the nine by-elections of the Sunak era is around 15%. Using Herdson’s thumb, that implies that polling should be showing us a swing of 10% or so from the Conservatives to Labour. The gap between the two main parties at the last election was 43:30, so that would imply that the two main parties currently stood at something like Con 33 Lab 40. That is much closer than actual polling is showing us.

Labour would protest that you could hardly treat Somerton & Frome as indicative of Labour prospects at the next election, and that Rutherglen & Hamilton West belongs to a different story entirely. They would have a point. Nevertheless, keep that naive approach in mind. It has the great merit of not requiring other value judgements being taken into account. There is no certainty those value judgements are correct.

We might instead look at the average swing away from the Conservatives to the eventual winner (regardless of party). That’s just over 17%. That implies that polling should be showing a swing away from the Conservatives of just over 12%. That would suggest the two main parties should be on something like Con 31 Lab 42. Only More In Common are finding anything like this.

Value judgements

Having cautioned about the risks of adding value judgements to raw information, I’m now going to take exactly those risks.

My preference is to segment these by-elections into three: Labour defences; Conservative defences; and Scotland. My rationale for doing so is as follows. In contests in safe Labour seats at a time when Labour are ahead in the polls, there isn’t the same motivation among voters to send the government a message as in Conservative-held seats, so we see lower swings.

We saw this in the 1992–97 Parliament, where the swings from Conservative to Labour in Labour-held seats were often far lower than in Conservative-held seats. Indeed, the swings in the three Labour-held seats in the last year are not far short of the swings in Labour-held seats in 1994 and much bigger than the swings seen in Labour-held seats in 1996.

The battleground in Scotland is quite different from that in England and Wales. Bar a couple of seats, Labour and the Conservatives do not compete directly with each other. Instead, each is battling the SNP in their respective areas of contention. Swing between Labour and the Conservatives is relevant only to the extent that it tells us about each of their chances against the SNP.

Conservative defences

That leaves five Conservative defences to consider. The average swing from the Conservatives to Labour of those five seats is just under 17% (equivalent to polling of Con 31 Lab 42 using Herdson’s thumb). The average swing against the Conservatives to any party is just under 21% (equivalent to polling of Con 28 Lab 45 using Herdson’s thumb).

The last of these looks more typical of current polling. We might decide that this is one way we might corroborate current polling. Or we might decide that the consistency between the two gives both more credibility. Or we might simply be misleading ourselves.

It’s becoming increasingly clear that Uxbridge was an outlier. Nevertheless, the Conservative strategists are right that it does tell them one important thing about how to campaign. The Conservatives need to shift from defence to attack in any election if they are going to stand a chance. So far they have failed to do so in any other by-election and they don’t seem to be doing so effectively at a national level. They’re trying, but so far it’s all too scattergun.

We’ve seen what can happen if the Conservatives fail in this attempt. A by-election swing of just under 24%, seen in Selby and Tamworth (both straight Conservative/Labour contests), equates to a swing in national polling of about 17%. The gap between the two main parties at the last election was 43:30, so a 17% swing would turn that into 26:47. That’s surprisingly close to the current polling average, which according to Sky’s poll of polls currently stands at Labour 45.8, Con 25.1.

(Given the confounding consideration of the Lib Dems in Mid Bedfordshire, I’d tend to put that to one side when considering the Labour/Conservative battleground that will decide the next election. As it happens, I regard that result as the very worst one of all for the Tories. In a seat with many Blue Wall attributes, Labour persuaded enough voters to help them oust the Conservatives even when they had the Lib Dems as a potentially viable option and where the correct tactical choice wasn’t obvious to all voters. They may struggle to retake the seat at the next general election now that voters know for certain who the best anti-Tory candidate is. There are quite a few seats in the area where Labour are second placed. Hertfordshire in particular could provide some interesting results next time.)

There is nothing in the by-election results to give the Conservatives any sense that their position is starting to improve. If anything, their position is deteriorating. Voters in by-elections have got behind the candidate best placed to defeat them on every occasion. It looks likely that we are going to see tactical voting against the Conservatives on a scale not seen for many elections. They are heading for a decisive defeat.

Scotland

We have a sample of one in Scotland. Using the same rule of thumb, that by-election is consistent with a 14% swing in Scotland from the SNP to Labour. That implies a national Scottish live split of Lab 32 SNP 31. That’s a bit more favourable to Labour than current Scottish polling suggests. But it’s a sample of one.

What about the local elections?

I wrote about this previously. The Tories’ national equivalent vote share was 26%, which was terrible and towards the bottom end of their polling at that time. Labour at 35% were far below their polling, but these elections were fought in areas where they were not locally strong, and the anti-Tory vote could and did coalesce round others. The Conservative tally (and their consequent seat losses) seems much more significant to me in that context. The Conservatives looked stuffed then and things have not improved for them since.

Summary

Taking all the evidence together, it seems that the swing against the Conservatives now is greater than it was in 1996 and the drop in Conservative support is greater than it was then. Taking all the evidence together, my sense is that Labour might surprise on the upside rather than the downside.

Other observations

The Conservatives’ election machine seems to be grinding its gears. Their expectation management in May managed to underplay the actual number of losses. Similarly there were press reports before Tamworth and Mid Bedfordshire that some Conservatives were expecting to win those seats. That suggests that the Conservatives’ antennae are not functioning very well at present.

Separately there have been reports that the Conservatives are being outgunned by numbers of Labour activists. This has the potential to make a substantial difference when getting out the vote.

Nowcast

The net of all this is that my sense is that if there were an election tomorrow, Labour would get in the order of 425–450 seats and the Conservatives 110–135 seats.

Futurecast

Of course, there isn’t going to be an election tomorrow. The Conservatives have a year or so to turn things around. Even though the Conservatives are still falling in the polls at present, I do expect this gap to close. Many current “Don’t Knows” will return to the Conservatives, out of muscle memory as much as anything else. Otherwise, however, the jelly seems to be setting. The public have made up their minds about the Conservatives and unless something makes them take fright about Labour, it looks as though we’re just waiting for the clock to tick down.

There is always the possibility of something unexpected. So far in this Parliament we’ve already had Covid, the invasion of Ukraine and the Hamas attack on southern Israel. The fallout from all three will reverberate right the way to the election and beyond, and the parallel US presidential election will also become entangled with British politics. No doubt there will be other shocks and surprises to come.

It’s far from clear, however, that any shocks or surprises are going to benefit the Conservatives particularly. It’s just as likely that they further undermine the government. For example, the Covid inquiry looks set to produce a stream of bad news stories for the government in the coming months.

So it seems reasonable to assume a central point in the final outcome that is based on the current trajectory. How many “Don’t Knows” will return to the Conservatives? Here we just have to guess. My guess is about half of them. I regard that as fairly generous to the Conservatives: I note that Opinium, who make assumptions in their polling that “Don’t Knows” will return to habit, still only record the Conservatives in the high 20s. Also, there are a fair few 2019 Labour voters who are “Don’t Knows” in the polls at present. Presumably a lot of them will return home by the general election too.

Even so, that should be enough to reduce the central point of Labour’s eventual seat tally to 375–400, bumping the Conservatives up to 160–185.

How to treat this futurecast

With extreme caution.

My nowcast is a lick of the finger given the obscure and limited data. My futurecast — relating to an uncertain date with uncertain events and uncertainty about how the public will react to those unknown developments — is still more imprecise. The range of possible outcomes remains high.

There’s a new market on Betfair on how many seats the Conservatives will lose. This starts at the 2019 total of 365 rather than any notional figure on 2024 boundaries, which would be higher. Even so, you can back 201+ losses for 2.78 at the time of writing, and this seems to me fair value right the way to 2.4 or even lower. I’m not the first to have spotted this. I am always willing to copy a good idea though.

It’s better to think of my guess as a bell curve with a low peak around the centre of those numbers rather than a limited band. I certainly haven’t ruled out the possibility that Labour fall short of an overall majority.

Quite apart from anything else, as I have already noted I am dubious about the reliability of current polling. The risk of a polling miss is always substantial and looks more so than usual just now. This could work either way and while my reasoned hunch is that this time it would be more likely to be in Labour’s favour than against them (if Reform voters don’t actually exist and Green voters are really Labour voters expressing enthusiasm for the Greens, that’s probably worth a few percent to Labour), that’s just a hunch. I have no backing evidence for my reasoning. A miss could be a miss either way.

Belatedly the markets have caught up with my long-held view that Labour should be very short-priced for an overall majority. I have for some time rated a Labour overall majority as something like 1.3, which is more or less exactly where the Betfair market is now.

I’ve set out my views. They’re tentative and based on skimpy information, but that’s what we’ve got. If you disagree with me, I hope I’ve at least made it easy for you to work out where you disagree with me.