Can you trust ConservativeHome's surveys?

The ConservativeHome Members’ Panel is increasingly being quoted in the media as the best guide to opinion within the Conservative Party.  On Wednesday night – at the party leadership’s request – I gave a short presentation on recent Panel findings at the Built To Last roadshow event.

Earlier this week a few visitors to this website (on this thread and on Guido’s blog) questioned the representative nature of the Panel.  Many of the attacks focused on the fact that many comments on this blog are hostile to aspects of the Cameron project.  I make no claims for the representative nature of the comments on this site.  I do believe, however, that the opinions of the Panel can be trusted and offer reassurance on three levels:

Track record. Last December 69% of the first 848 first members of the Panel said that they would like David Cameron to be the next Tory leader and 31% said that they hoped David Davis would become Tory leader.  The actual result was 68% to 32%.  This group of 848 people continue to be the bedrock of the Panel.  Nearly all continue to participate on a monthly basis and I believe that they are a good guide to Tory opinion.

Weighting.  We compare the first question of the poll (which is always the same) in the previous and current surveys and take as our reading the actual change within that group (the matched group is about 95% of all respondents). We also match current respondents to the original 848 respondents as a control. Having weighted the first question, we then weight all subsequent questions to that first question. So far, weighting has never required more than a 1% adjustment, as the panel is in fact very stable, with new respondents closely resembling the profile of the earlier respondents. (Supporters of neither candidate in last year’s leadership race are proving to be disproportionately slack or enthusiastic at participating.)

Recruitment.  My reason for believing that the Panel members represent the membership is the routes via which we have been recruiting members.  The Cameron and Davis campaigns both emailed their supporters in order to alert them to the existence of the survey, and party organisations like the Conservative Christian Fellowship, Conservative Way Forward, Conservative Friends of Israel, Tory Reform Group and Women2Win were invited to alert their supporters to the surveys.  Since then I have emailed Tory Associations and leafleted Conservative events.  As I leafleted the Spring Forum in Manchester it was striking how many of the people I invited to join the Panel said that they were already signed-up. One possible criticism which I do accept is that the Panel represents activists rather than the full range of the non-activist membership. However, there is no evidence that non-activist (or non-participatory) members are essentially different in their views from activists. Any evidence about this is virtually impossible to obtain. It would therefore be wrong, and indeed meaningless, to attempt to draw any conclusions about the views of non-participatory members.

I do not pretend that the online survey of the kind being run by ConservativeHome is perfect.  There are probably some people who vote in the surveys who are not paid-up Tory members and there may be some infiltrators from other parties.  But the three reassurances above will, I hope, offer readers of the survey a sense of the Panel’s overall reliability.  Over time I hope to add one or two more security measures.

While we are on the subject of the Panel it might also be helpful for me to restate why ConservativeHome runs it at all.  The survey results don’t always make comfortable reading for the Tory leadership or for individual shadow cabinet members.  Some of the visitors to this site who are most loyal to the current party leadership believe that the survey provides ammunition to the party’s opponents and I am not insensitive to their concerns.  I, for example, delayed publication of the April dip in satisfaction with David Cameron because I did not want to encourage other parties’ council candidates on the eve of May 4th’s local elections.

The reason why ConservativeHome will continue to survey the Tory grassroots/ webroots, however, is our fundamental belief that the party is not owned by the current leadership or any leadership. Gone are the days when CCO can purport to speak on behalf of members to the press without fear of contradiction.  ConservativeHome believes that the leadership deserves loyalty and respect from the grassroots but loyalty and respect is a two-way thing.  The party must listen to the grassroots.  We are the people who raise the party’s funds and deliver the leaflets.  ConservativeHome’s first campaign was the successful championing of the membership's right to a vote in the leadership election and, much more recently, the membership’s right to be able to inspect the candidates’ A-list from which the next generation of Tory MPs will be drawn. ConservativeHome’s mission involves helping to build a more open, democratic and participative Conservative Party.  The voice that the Members’ Panel gives to the grassroots is a fundamental part of that mission.

***
Recent Panel findings: David Davis tops satisfaction league table, state funding of political parties rejected by 81% of Conservative members and 54% of Tory members say the war in Iraq was a mistake.

Comments

[Wat Tyler]

Go Tim

ConservativeHome has become a national institution. As a counterblast to the in-built centralising tendencies of senior management it's value is well beyond rubies.

Of course, the surveys may not be as scientific as the mainstream polls like to claim they are. But they seem to me to capture precisely what most grassroots Tories are saying round our way.

Keep it up.

[Editor]

Thank you Wat.

One funny story about the Panel...

I believe that a good number of Tory MPs are Panel members and, just possibly, a few of the ten senior shadow cabinet ministers we regularly rate. One of those ten recently joked to me that he/she doesn't vote so much as to boost his/her own rating but to vote down the rivals!

My lips are sealed as to the identity of that individual!

[James Morris]

Tim

Great Stuff. It would be good though if we could develop mechanisms for getting opinions on the Conservative Party across the Conservative coalition in the broadest sense. As we have discussed before the nature of party political affiliation has changed and is changing and though your survey of members is incredibly valuable it would be good to find ways of connecting with opinion outside the clearly defined boundaries of membership.

[Henry Edward-Bancroft]

Wouldn't it be great if we could back up the excellent work done by ConservativeHome with greater fundraising efforts? Now let me think... ;)

[matt wright]

While there are some people who post on this site who are just negative or anti DC, I have to say that on balance this site is a great benefit to conservatism. The panel is generally representative of party members and it shows the very strong support DC has. I know this is reflective of the position amongst members as when I attend association events, a large majority are strongly behind DC but there are a few loud sceptics. In some ways the ability of my party to engage with this modern form of debate is a test of its wish to be more local. I like the way that recently Francis Maude has posted to the site and got involved. In any party or amongst its supporters there are different views, that is just life. I face the same amongst my constituents as a councillor or colleagues at work. What sensible people do is listen to all this and in some cases change their approach or in other cases reflect on this and explain why they will not. Change is not a fixed plan it is about growth,

Matt

[Zhukov]

I think a lot of the comments, certainly mine, on Guido's site were less about the surveys and more about the comments being left on the site.

Although Conservativehome is a fantastic resource for news the comments section has become a forum for oddities like William and Chad, people with their own narrow agendas.

[Chad]

for oddities like William and Chad

LOL

* Against state-funding of political parties
* Against positive discrimination
* pro-grammar schools
* pro EU referendum.
* pro low tax environment

Such beliefs used to be called 'conservative' but if they are now called 'oddity' then I am very happy with that label!

[Henry Edward-Bancroft]

If I did have a criticism it would be that shared by "Zhukov" the discussion does tend to be monopolised by a handful of people with entrenched views.

While I wouldn't go as far as to call them "oddities". If there was some way of broadening out the discussion that would be helpful and, i think, aid the quality of debate.

[Chad]

Henry,
Would you care to provide an example of one of my 'entrenched' ideas that do not stand up to rational criticism?

The last time I checked, 81% of conHome members agreed with my opposition to state funding of political parties.

I would suggest that it is the party leadership who have entrenched views.

By 'quality' I guess you mean agreeing wiht Cameron.

[esbonio]

Well said, Tim.

Whilst not a regular poster on your site, I have been reading it and making occasional comments from its inception. I find your site excellent in all respects and believe you provide an invaluable service. Your aims and your methodology seem fine to me. Please continue the good work which includes allowing a variety of opinions to be expressed.

[Henry Edward-Bancroft]

Chad,

I certainly don't disagree with much of what you have to say and often quite enjoy reading your posts.

[James Burdett]

I find this site interesing but agree with Zhukov up to a point. There do seem to be a heck of a lot of indivisuals on here who want to occupy certain idealogical ground regardless of how broadly attractive it is. Many seem to hold a 'mountain must come to Muhammed' tendency and not see the postive virtue with engaging with the electorate where they are not where you would like them. It is particularly depressing that many seem to be the younger posters, the older hands seem at least to see the virtue of pragmatism.

[Editor]

Chad et al: I'd be grateful if this thread didn't become a debate about the qualities of the various people who leave comments on this site.

[Jonathan Sheppard]

What is that phrase about if something didnt exist we would have to invent it... well that's true about conservativehome. One can't imagine the control freakery in New Labour allowing something like this site to exist. Tim and Sam deserve a good pat on the back for all their hard work. I'm sure this site is going to go fron strength to strength.

As for people suggesting the site is dominated by a few regular posters. Well you know what to do about it... express your own views as well.

[James Maskell]

I see the teams of those key critics has changed again. Ive been booted out with substitute William in now...damn. Who do I have to screw around here to get back in the team....wheres my diary secretary?

[Chad]

Tim,
I agree, I only respond to personal abuse that is not deleted. A right to return seems fair in such circumstances.

I always ensure my posts focus on the issues, but the personal abuse that comes back if more often than not left, then I am criticised for simply challenging those who attack the person not the issues.

[William Norton]

Zhukov:oddities like William and Chad, people with their own narrow agendas

Which William do you mean?

I assume you mean the nutter who keeps suggesting the far right are going to eliminate the Tory Party?

I'm the nutter who keeps asking pedantic questions (e.g. "why have you misspelt 'Marshal Zhukov' in your URL?", "which William do you mean?" that sort of thing). I would have thought pedantry was quite a broad agenda, although strictly pedantry is more of an attitude and doesn't have an agenda as such because that connotes ..... etc. etc.

[Editor]

No problem, Chad. It's only fair that you respond to attacks on you.

[malcolm]

Tim & Sam you both hopefully already know how much I admire the work you have done on this site.
To those posters who complain that people who disagree with the party leadership post too often I would argue that stimulating debate and arguing ideas is the whole point of this site. Wouldn't it be be boring if we all agreed with each other?

[Patsy Sergeant]

James Morris @ 9.35 - Well here is one member of the public who is NOT a member of the conservative party , at least in the joined up sense, but I think in my various postings I have nailed my allegiances to the relevant post! And the Editor can vouch that I do contribute.

What I am saying is that just being a 'member' doesn't necessarily mean that the person is involved; similarly just because people haven't 'signed up' doesn't mean they won't vote conservative when the time comes (I realise it would be much nicer for CCHQ if they could have everything tabulated and quantified beforehand!)

I think the most important thing is to get more and more people involved here on the internet.

[James Morris]

Patsy

Yep its good to see people who are not members of the party contributing to the discussion. We need more of you on this site. Also, the point I was making in my previous post was that we need to actively seek out dialogue with people who aren't members but may feel in some way affiliated through their work, voluntary work or just their personal values with what we are trying to do as a political party.

[Ted]

Agree with James Morris about bringing together party sympathisers - and I think this is happening as CHome has become better known

On representation I think that Tim's methodolgy while not perfect does show that that the monthly polls are roughly reflective of the party overall (except in perhaps the views of indivdual performances where the more activist membership here probably has more entrenched views and longer memories!).

[kingbongo]

As one of the guilty posters on Guido I should say I think CH is a very valuable resource. Francis Maude explaining to conservatives via CH his thoughts on the A list and so on are really good.

I do place less faith in the survey results than I did initially but that's only gut feeling.

I think there are many frequent posters who seem to have only one interest and that is to slag off the party leadership and turn every thread into a debate on the evils of the EU (which are legion, but shouldn't be the backdrop to every discussion).

I know this is a site independent of the party but it is seen as a place to find out what party activists feel. Anybody reading a Goldie, James Hellyer or Christina Speight post would think civil war is alive and kicking in the party when it isn't.

A particluar problem I think is crazy posts from the likes of the William who believes elections are decided by a cabal of secretive figures in Brussels. For CH to be taken seriously this sort of thing should be stopped.

William Norton's Film Reviews are of course to be encouraged as is Chad; provided he promises to change his advert soon as the funny ones are much better.

[Chad]

as is Chad; provided he promises to change his advert soon as the funny ones are much better.

:-) I agree, that one is boring. You won't have long to wait though KB. I've been busy updating a new version of the site, and the new ads will be (well attempting to be) humorous and a little bit racy too...

[Graeme Archer]

Chad is too determinedly good-humoured to take against! I may form a non-Chaddites for Chad party.

Sometimes I'm reading the comments and I think "oh god not X ranting away again" - so I slap myself and reflect:

1) Physician, heal thyself;

and anyway

2) I only keep reading the posts because I keep coming back and reading the articles. It's like when you're in a pub and you say "look at that loser, he's always in here" -- doh.

[Daniel Vince-Archer]

"One of those ten recently joked to me that he/she doesn't vote so much as to boost his/her own rating but to vote down the rivals!

My lips are sealed as to the identity of that individual!"

I suspect Jack Stone will soon pop up and blame David Davies (sic), William Hauge (sic) or Liam Fox.

Seriously though, I'd like to echo the warm sentiments expressed above about the value of this site.

[buxtehude]

If people like Chad are 'oddities' it is because it is odd to debate issues openly and fairly, as he does.

Just imaginbe what this site would be like if people were NOT pursuing their own interests and obsessions. Isn't that the whole point? That we get a richer debate about subjects through the engagement of the oddities that are in all of us?

What would be the point of a blog for Political Stepford Wives?

[Gareth]

There's no reason whatever to doubt the accuracy of the surveys. In fact, I suspect that, if anything, the surveys under estimate the unease of party members at some of the Cameron reforms.

[Jon Gale]

Chad "The last time I checked, 81% of conHome members agreed with my opposition to state funding of political parties."

But do you have to bring it up in every thread?

[Chad]

Jon,

:-)

I bet you £1000 I haven't brought it up in every thread. Will you take the bet?

In fact, I'm sure you can't fund another today that it is mentioned.

[Deckchair of despair]

Tim or Sam:
You say There are probably some people who vote in the surveys who are not paid-up Tory members
I am not a Party member (nor have ever claimed to be), although I know people who are. I therefore always tick the "not a Party member but a Party supporter" box. What happens to all my carefully considered responses to your survey questions?

Incidentally, I think this site is one of the very best on the web. Although I haven't myself posted much recently, I read it very regularly, as I have done almost since its inception. I particularly like the mixture of practical considerations and theory which is generally found here, and which is almost unique.
Incidentally No 2: I'm not really anonymous. It's easy enough to find out my name. I only adopted the dopey soubriquet I use because on a couple of other sites where I have posted comments in the past, there were a couple of regulars with similar names to mine, and I was simply trying to avoid confusion.

[William Norton]

With all the various Williams, Robs, Richards, Jameses etc. - have we at last found a justification for ID cards? It's about the only theory the Govmt haven't floated yet.

For all the methodological queries which could be raised against the survey (and either Guido or Graeme Archer are better placed to expound on those) it does pass the Ronseal test: it does what it says on the tin. I'm quite convinced that it reflects accurately the views of members/supporters: at 1,500 responses it compares very favourably to a national opinion poll (although it shouldn't be judged by the same criteria).

My view is, has been and will remain that the survey is a good guide to the views of members and is a fair guide to the sentiments of supporters, and the message-posters are a fair sample of activists of various shades, notwithstanding (or perhaps precisely because of) the more, ahem, focussed viewpoints of some of our colleagues. Unfortunately, the people who are active in political parties - any party - will have a proportion of anoraks and fruitcakes. Why, just the other day I came across a chap who had formed a party and named it after a John Lennon song. I mean, how sad is that?

[Graeme Archer]

Thank you William (NORTON! WILLIAM NORTON!). I'm not really qualified to speak on anything. I have been thinking about Tim's explanation of how he weights the panel results but honestly I know nothing about survey methodology. Clearly it's not a random sample - which makes inductive reasoning difficult (to say the least).

Tim you actually have a rich data set. It might be worth looking at the within-subject temporal correlation on Question1. If it follows a definable pattern, you might be able to improve imputation for the missing values in any survey (why people are missing is (1) unknowable but (2) important). Also as a sensitivity analysis, would be nice to look at histograms from members vs non-members, missing last month but present this month vs present last month and present this month &c &c. You should contact a university sociology department and offer them access in return for a donation!

The panel survey however doesn't seem to break any obvious basic "rules" (unlike the Poll of Polls, which [ descends into blithering, repetitive red-eyed rage about the FOOLS who wouldn't listen, how I'll be justified in the end, as ye shall sew, would be better to replace it by asking passengers on the no. 48 (a poll of Poles, geddit) &c &c ... ] ).

[William Norton]

Graeme: you may claim modestly to know nothing about survey methodology, but in the great spirit of Conservative Home that shouldn't stop you from opining on the subject, or indeed being the best qualified person here to do so.

After all, in the words of the old Chinese proverb, he who does not know, and knows that he does not know, um, knows that he knows that he does not know. Or something like that. You get my drift.

[Graeme Archer]

Is anyone going to the Local Election Review event/drinks at CCQ on 30 May? Just bought two tickets ... lucky Mr Keith. Imagine ((c) Chad) how he managed to get through life before he met me, without ever having to stand for election, deliver leaflets, attend thrilling meetings in sweaty little rooms, was able to watch a football match from start to finish without having to bother with Alan Hollinghurst (shameless plug for review), had NEVER HEARD OF BELLE AND SEBASTIAN! Anyway. I live in the continued hope that I might finally meet some Conservative Homies Sensual Moyenne other than that John Moss non-virtually.

I have seen William Norton twice actually. First time, he had a map of North East London, and a bag of Cluedo pieces. It was sheer genius. I don't think Boff followed it though William.

[Henry Edward-Bancroft]

Getting some academic advice, as suggested by Graham is not at all a bad idea.

It might give the site and the figures a certain credibility if they were academically analysed.

[Richard]

"Just imaginbe what this site would be like if people were NOT pursuing their own interests and obsessions. Isn't that the whole point?"

The problem is that it often leads to threads being derailed and completely different issues discussed. There's a difference between going off on a tangent related to the subject and trying to introduce a completely different issue.

Then there's also the fact that some of us get bored with repetitive posts saying the same thing over and over again.

[Chad]

Personal attacks are the most regular cause of tangents and thread disruption imho.

If there was a strict 'attack the issues not the person' (so no more "oh you would say that because.." etc) approach then we could keep the threads more focussed.

Personally, I'd like to see registered members, even if they have to hide their name for work reasons, and to see more people entering profiles in the community section so we can know more about them.

Either way, the site works incredibly well, and long may it continue as an independent voice of conservatism with of course a heavy focus on the conservative party.

As we saw with the election night seat prediction, the range of opinions does seem to bring neat balance and accuracy.

The only minor concern for me would be if Tim did actually go back to working within CCHQ, as it would be incredibly hard not to keep the site independent, and if the balance skews then it would lose its relevance.

However, for now, conHome is the best place to discuss (small c!) conservative issues online, and long may it reign.

[a-tracy]

Buxtehude: "What would be the point of a blog for Political Stepford Wives?" at 12:54

There would be no point and no interest in a ra ra brigade site. If posters can't answer 'other posters' points of view, from a conservative perspective, then how will they go on with floating voters on the doorsteps. Chad, who seems to bring on the vapours, (amongst others), gives the Conservative Party activists an opportunity to test out their arguments to a receptive audience whose basic views are in line with their own.

[Annabel Herriott]

I read CH daily, keeps me in touch, but I dont post unless I feel inspired to write something relevant or useful.
What I absolutely do not get about Chad, is how on earth does he pay his household bills? He is posting morning noon and night. Is he on benefits? Is he self employed? Does his boss never check his PC? Does he have a home to go to? The musings are endless, and continue to give me something to IMAGINE!

[Editor]

Responding to James Morris (09:35) - I'm completely with you on the need to reach out to non-members. The future of the party depends upon us moving from a party of mass membership to a party of mass connections. Members still deserve a special role in the party's affairs but non-member supporters should be increasingly important in influencing/ deciding candidate selection and fundraising etc. In an ideal world the monthly survey would be as interested in recording the views of non-member supporters as members but the number of such folk completing the survey are currently too small to be very reliable.

What I will do, Deskchair of despair (13:27), is at least find ways of publishing the non-members' results but, because of the limited number of responses, I wouldn't want readers to interpret them as authoritative.

[Matthew Oxley]

Hi,

I often find those who complain most often about the quality of the poll tend to be those who don't like their results.

There are some findings that irritate me too, like George Osborne being rated so low, but these are the opinions of a large, and reflective (as Tim has already pointed out) amount of people, and thus deserves respect.

I can't say I was thrilled with the results of the last 3 general elections either, but whinging about how unrepresentative and unfair the polls were (90,000 more people in England voting for us yet us returning almost 100 less MP's than Labour) won't get us anywhere, will it?

[Patsy Sergeant]

I will tell you something... I have found it so exciting to be part of this website over the last few months, it has literally given me a new outlook on life, and helped me overcome a grievous loss.

I have learned and am still learning an enormous amount about conservatism in general, and I suppose you could say also political philosophy (whatever that is!), but above all I am learning yet more about how people's minds work, which is eternally fascinating!!

I haven't explored the internet a great deal but this is by far the most sophisticated website I have explored yet, and I haven't explored half or what is available here yet!!!

It may sound frivolous, but it is such fun!! I do however realise that there is a serious side......

[David Belchamber]

I find this a fascinatingly interesting site and I believe it can serve a vitally useful purpose by encouraging total freedom of expression about the way in which DC is taking the Party. It is an exciting time because everything appears to be up for grabs. Party policy is in a state of flux.
Although we all have our own interests and obsessions, I imagine that almost all contributors could sign up to a fundamental set of what we would regard as conservative values (security of the realm, law and order, smaller and more devolved government, less bureaucracy, lower taxes etc).
What interests me more than the detail of the policies currently being formulated is to try and determine how DC and his team are going to behave when in government. I do not want to vote for another PM only interested in his own legacy, I want a government team that will "govern at all times in the best interests of the whole nation". That requires honesty, integrity and accountability, all qualities that have been in short supply over the past few years (and not only in New Labour!).
Can DC and his team actually manage to run anything? The present government couldn't even manage to organise the proverbial party in a brewery. The shambles in the Home Office (as Michael Howard pointed out recently in The Telegraph) is a matter of process. So, too, is the NHS. They both need management of the highest order (you could add several other departments as well).
Are we safe to believe that all our legislation will be returned to Parliament to enact after full and proper debate in the H of C and not picked up by the experts only from statutory instruments at a later date?
Is there any chance of only having new legislation that is (i) equitable and (ii) based on common sense and natural justice?
What are the chances of returning to a situation where the judiciary and civil service are clearly distinct from the government, where spin doctors are consigned to outer darkness and we receive only factual information from press secretaries about the government's actions?
There are many exciting policies being worked out and DC is to be congratulated on insisting that only long term solutions should be sought.
We can help by expressing our views as ordinary (but very concerned) people who are willing the Tories to get it right this time.

[briank]

Where can i find this "Panel" and/or its findings/views?

[ramon newbery]

The Tory party will do this the Tory party will do that er excuse me but has anyone bothered to ask members and supporters what they would like to see the party do. NO.
I have a long list if anyone`s interested. Probably not. Oh well who cares. I`m just a voter who is only wanted for my vote. Which beggars the question; who is the master and who is the servant. Shall I serve the drinks now sir?

[Helen Barker]

I know for a fact that the majority of the conservative grass roots are against Britain's continued membership of the EU. Why dont you carry out a party referendum? It is sad that all the conservative policy relating to the EU is made by just a few EU enthusiasts at the top.

[Anthony Fontes]

Can you persuade most of your correspondents to be brief, terse and to the point. I have not read most of the contributions chiefly because they are boringly long

[Gerald Pym]

Bilderberg Steering Committee Member Clarke To Lead UK Conservative Party?
[Note: Ken Clarke's long-time position as one of the 30 man Bilderberg Steering Committee means he is at the heart of the conference agenda alongside Henry Kissinger and David Rockefeller. Clarke is one of only two Brits on the Steering Committee and decides who gets invited from the UK each year.]

[Duncan J Richardson]

I have particpated in surveys and have indicated that I am a member of another party. I suggest that you have categories for a) former conservtives and b) supporters of all UK parties. By this you could obtain useful information on the current allegiance of former conservaive voters.