Stephan Shakespeare: The Tories aren't doing better in the marginals

When polls are spun, we should be very nervous. The purpose of a poll is to provide independent evidence about what people think, evidence that may be used to measure progress or justify a particular strategy. When polling data is misinterpreted, it’s therefore worse than having no data at all, because it gives us confidence in a proposition that may be false. A recent analysis of voter preference in the marginal seats presents just such a danger.

The Conservatives have certainly pulled the polling numbers back impressively recently; a week or so ago the Labour lead stood at 6%-8%, and now the average is down to just 3% using the last polls from ICM (5%), YouGov (3%) and Populus (1%).

However, an analysis of the three most recent Populus polls, separating the data from just the 120 most marginal seats, painted a rather too rosy picture by wrongly suggesting a better result for the Conservatives in the seats that matter most, the 120 ‘most marginals’, prompting ConservativeHome and others to think that the targeted campaigning effort being made in marginal seats is paying dividends.

It may or may not be paying dividends – but the poll analysis provided no evidence for it. In fact, the correct interpretation of the data is that change in voting intention is exactly the same in the marginals as it is in the rest of the country. And that change is zero.

Am I saying the data published in The Times, was wrong? Not at all. The 4.1% lead it showed for the Conservatives in the 120 most marginal seats was indeed better than the 1% it showed in its latest poll for the country overall. But that is no surprise: the 120 most marginal seats contain many that Conservatives won in 2005. It represents zero swing.

Let us assume that the Populus polls are as accurate as it is possible for polls to be. Working back from the table published in Tuesday's Times, we see there was a 0.25% swing to Labour in the country as a whole, and 0.2% swing to the Conservatives in the 120 most marginal seats.

Bear in mind that even the most perfectly conducted poll, by whatever methods one thinks are the best, must still carry a margin of error when one or two or even five thousand people are sampled in place of the whole population. In the case of this marginal seat analysis, we have a sample of around 900 (one-fifth of the seats in three polls having individual samples of 1,500), with a margin of error of around 3%. By any normal statistical accounting, both 0.2% and 0.25% in these polls must therefore mean 0%.

So it is worthless to draw any conclusions – on the basis of this polling evidence – about the Conservatives’ performance in the marginals, and about whether this or that targeting strategy is working or failing.

Actually, we should be extremely cautious about any polling data adduced to demonstrate a different voting intention in the marginals. Consider the conventional voting intention question, something along the lines of: “If the general election was tomorrow, which party would you vote for?” If you were a Labour supporter, living in a Conservative-LibDem marginal seat, you would be very likely to say ‘Labour’. You will take the question as asking you for your party preference. You may have no idea that you’re living in a marginal. Of course, come the general election, you may realise that in your constituency, a vote for Labour will not achieve what you want, and you vote ‘tactically’. But that will rarely be picked up in polling.

A question we at YouGov have asked in the past - along the lines of “If you were living in a constituency in which Conservatives and Liberal Democrats were running neck and neck, with the other parties having little chance of winning, who do you think you would end up voting for?”, while also ascertaining attitudes to tactical voting – might give slightly better tracking data, but even then I would put little confidence in any conclusion predicting success for one party or another. One has no idea how well-informed the respondents would be on the day of actual voting, and how much they would be leaning towards tactics or commitment. This is a minefield for pollsters.

I hope I’m not sounding too fussy, but I do think it’s important: whenever we try to be too clever with insufficient or ambiguous polling data, we are likely to be fooling ourselves rather than informing our strategies. The desire to find the data you want to find is so fundamentally human, it afflicts even the most rigorous experimental scientists working in superbly equipped laboratories. It’s very hard to resist the temptation to over-interpret, especially when you yourself are involved in the game, and I certainly don’t suggest I’m not frequently guilty myself.

Comments

[Graham Smith]

That's a really helpful explanation, Stephan. Thank you for your kind assistance in interpreting the reality behind the headline figures.

[Annabel Herriott]

The only strategy that really works in a marginal constituency, is to get out there, and survey, survey survey.Your own doorsteps, your own issues, your own voters or otherwise. Then to analyse it properly, as well as dealing with the local issues that the surveys brought up.

[HF]

Stefan poors gold water on the view that things are going better (overall) in the marginals. Well done.

I have come round to the conclusion that we have not made enough progress to win a GE in next 18 mths and must adopt a 2 stage approach, by first winning the easier marginals.

We have to be realistic and acknowledge that Heffer/Daley's Telegraph and Dacre's Mail carry too many attacks on us and have influenced a couple of % of the voters. Throw in the look warm Murdoch papers and the biased BBC and we have a media challenge that Coulson will need a year or more to overcome.

That said we will win a lot of Southern marginals and some in the Midlands. It is most of the North and Scotland where the challenge remains and these marginals bring down the overall % improvement.

Lord Ashcroft has the right strategy and just needs to get on with it. Hague (North) and Goldie (Scotland) need to radically improve their game, or step aside.

[Effie]

My gut feelings tell me that Brown is further ahaead than even these figures suggest.
I do not think this report by Goldsmith and Gummer is going to do Cameron much good, so much so I think this will be reflected in the next round of polling.
People are turned off by new taxation.
A lot of people do not swallow the green issues as much as team Cameron would like to believe.

[christina]

This is salutary and adds to my constant plea that we treat the polls as reasonably accurate but that the interpretation put upon them as being often flawed, My instance is that the "headline" figures never show the Don't knows, the Won't votes or those who will vote but refuse to answer for whom, This sector is often bigger than any of the 3 main parties.

Shifts in the headline figure considered without the knowledge of movements in and out of the unreported sector. This can be crucial.

For a hypothetical example take an apparent Labour improvement. This could be caused not by a switch of Tory or LibDem voters but by a return to previous loyalty of Labour supporters who disliked Blair so much that they have been lurking as Don't Knows.

[Chris Heathcote]

This seems to me to needlessly pour cold water on the polls findings. It radically overstates the impact of tactical voting, which only ever motivated a small percentage of people and which has been in decline since a peak in 1997. It forgets that the vast majority of people who are likely to vote tactically and live in marginal seats will probably have already been voting tactically in the last three elections and will not decide on the last moment. Finally, there is the assertion; "the 120 most marginal seats contain many that Conservatives won in 2005" - surely this figure of 120 is the number of extra seats the Tories need to win to form a Government and therefore inlcudes only seats they don't already hold. And even if it does include Tory-held marginals, this can be at most 30 seats, meaning the vast majority are not.

[Scotty]

I did not think that the Populus polling data used a big enough sample, over a long enough period to be a definitive guide to what was happening in marginals. All the polling figures over the last few months seem a little volatile because of the uncertainty and changes in the political landscape.

[Scotty]

"My gut feelings tell me that Brown is further ahaead than even these figures suggest."

"People are turned off by new taxation."

Effie, if your argument holds water then this is most damaging to Brown because he has a reputation for introducing stealth taxes year on year.

[Scotty]

"When polls are spun, we should be very nervous."
One point I forgot to mention in my earlier post. Every newspaper that commissions a poll, whether monthly or less frequently tends to "spin" the results to suit their agenda.
IIRC the NoW did just that with a poll that gave the Conservatives a comfortable lead, the headline and the accompanying analysis bore no resemblance to the figures reported.

[Old Hack]

Its interesting to see the criticism of YouGov on PoliticalBetting.com

In particular YouGov's failure to weight data by the respondents propensity to actually vote.

http://politicalbetting.com/index.php/archives/2007/08/31/labours-yougov-lead-down-a-touch-to-8/

That said, much of what Mr Shakespeare says here sounds reasonable. One doesn't need a pollster to note that the word on the street that Brown has made a good start as Leader.

My guess is that the much more subtle effect of Brown is that he may well motivate disgruntled Socialists to turn out and vote, though I don't think he'll attract much if any new support to Labour.

However given the current state of affairs that may be all he needs to deliver a working majority?

[Mark Senior]

I believe the breakdown of the 120 seats was something like 44 Con 56 Lab 17 LibDem 3 SNP 1 Galloway - yes I know that totals 121 LOL .
The misuse of poll results to try and prove something that they do not actually show seems to be ever growing .
The Use and Abuse of Statistics by Reichmann published in the 1960's by Penguin and Darrell Huff's How to Lie With Statistics published in 1991 are both useful in showing how both the media and parties spin poll results falsely .

[Effie]

Scotty | September 10, 2007 at 11:27 AM

Just step aside from red/Blue politics for just a "Mo" and consider this:

Everybody accepts that taxes have gone sky high, nobody disputes this.
However Brown Et-Al were extremely clever in getting through to the electorate that these taxes were needed for Schools-NHS etc, the general public understand this.
Now I am not debating the rights and wrongs of this policy, people have their own take on these issues. What I am suggesting is most people do not see the green agenda in the same light and do not attach the same importance to it and it is going to hurt Cameron more than GB.
Brown's policy right-wrong or indifferent has been accepted, most people I know are not so convinced of the green issue. They just see it as another excuse for the politicians to tax us some more.

[Somewhere]

Effie, I can assure you as a candidate who has spent weeks and months out on the doorsteps that your gut feeling is certainly not correct.

My factually based feeling is that Brown is doing WORSE than the polls suggest. But what would I know, I'm just out at the sharp end.

[Stephan Shakespeare]

I'd like to just emphasise what I said above, "The Conservatives have certainly pulled the polling numbers back impressively recently". It was not my intention to pour cold water, only to make sure that genuine gains are seen with clarity.

I know the party has a sophisticated polling operation and nothing I said in any way casts doubt on it - only on the media's over-eagerness to find a new angle on some polling detail, which of course all newspaper inevitably do.

I have no doubt that the party's pollsters are fully aware of the difficulties, and of course they are in possession of much more data than we are able to see from the outside.

Chris Heathcote - no, the 120 is not the ones most needed by the Conservatives, it is all the most marginals and the figures I use on swing are taken from those that were published.

"It radically overstates the impact of tactical voting" - I'm not saying I know what the effect might be, only that we can't tell what it is. But clearly it does have an effect which is not measured by the conventional polling question at this distance from a campaign.

[Cllr Brook Whelan]

I feel the polls do not take into account factors on the ground such as if there is an outstanding local candidate in the area, or big local issues. They only ever seem to put a spotlight on the leaders like a presidential election.

Therefore I don't think polls should ever be read too seriously.

Voters are also very fickly. If they are asked how they will vote one week, and they are asked again the following week, they will quite often give two conflicting answers.

By the way, there is a very good article on Nick Robinson's blog on the BBC news website.
It is about the 'lurching to the right' tag.

[Effie]

Somewhere | September 10, 2007 at 01:13 PM

I certainly would not dispute your findings, however it depends on the part of the country you come from.
Allow me to inform you Mr Cameron is making no inroads what-so-ever in the Midland-North and Scotland.
Can I also draw your attention to the last two bye elections that of one in the North and one in the South where Mr Cameron was pushed into third place behind Ming Campbell.
I know a lot of Conservatives point to the local elections and that is where they go wrong, we voted Conservative in those elections to get our dustbins emptied on a weekly basis, we would never vote that way in a GE and I am sure we are not alone.
I have a good many friends many of whom like myself are political animals, not all dyed-in-the-wool Labour supporters by any stretch of the imagination.
Not one of them is being won over or convinced by Mr Cameron, they see him as vacuous, shallow, flip-flopper.
Plus the fact the disloyalty shown in recent days by Bercow and Mercer not to mention a certain Tory grandee, one Michael Ancram an ex party chairman this in itself does not make good reading and displays a certain ammount of division within the party.
The electorate hates this and Labour make a better job of concealing the skeletons in the cupboard than the Conservatives do.

[Somewhere (North of Nottingham)]

Well Effie - As I'm in a northern marginal I beg to differ, but then what would I know...

[Somewhere]

"Vacuous, shallow flip-flopper" - it worked for Blair ;)

[Scotty]

"Not one of them is being won over or convinced by Mr Cameron, they see him as vacuous, shallow, flip-flopper."

Effie, lets look at the recent record of one Gordon Brown whose vacuous and shallow PR stunts are already marking him down as a serial flip flopper. Referendums, super casion's etc etc......

[Chris Heathcote]

I'm amazed at how 'Effie' is singly able to assure us that where they live - a place called "Midland-North and Scotland" David Cameron is uniformly seen as a "vacuous, shallow, flip-flopper". Faced with the choice of an informed poll and a sweeping partisan statement, I know which I'll believe...

[eugene]

The famous 'margin of error' can work the other way...and we could be even further ahead in marginals.

I think we should work hard but take the poll as reason for being optimistic.

[Effie]

Chris,September 10, 2007 at 04:50 PM

I live in the Midlands, I have family living in the North and in Scotland. It may come as a surprise to you but people do live North of the Watford Gap, they have opinions as well as a vote.

Sorry Scotty that is not the way GB is being seen.
Just look at his ratings against Cameron's as to who would make the strongest PM.

[TomTom]

I don't think people in Northern England like Gordon Brown; yes they think Cameron is shallow and irrelevant, but Brown is seen as alien too.

Frankly politics is beyond the pale for most and they see it as squabbling contestants on a game show, irrelevant to their lives and determined only to screw up things and impose more immigrants and taxes on them in their daily lives.

Cameron, Brown - it is a political class - like deciding whether you prefer Khrushchev or Brezhnev - same party same government - same oppression.

It is more an interest in sell my house, go abroad than party politics. I expect a hung parliament because people want no overall control

[Somewhere]

So Effie, you dont actually live in the North so perhaps you could stop commenting on behalf of all of us who DO actually live in the North!

[Chris Heathcote]

Effie; I could say exactly the same; I'm from the Midlands, and my family live in the North and in Scotland, but I dont feel that makes me singly able to make sweeping judgements on complicated political trends. Let's have some real analysis of this poll and real local examples if people dont think it's working...