Can you trust ConservativeHome's surveys?

The ConservativeHome Members’ Panel is increasingly being quoted in the media as the best guide to opinion within the Conservative Party.  On Wednesday night – at the party leadership’s request – I gave a short presentation on recent Panel findings at the Built To Last roadshow event.

Earlier this week a few visitors to this website (on this thread and on Guido’s blog) questioned the representative nature of the Panel.  Many of the attacks focused on the fact that many comments on this blog are hostile to aspects of the Cameron project.  I make no claims for the representative nature of the comments on this site.  I do believe, however, that the opinions of the Panel can be trusted and offer reassurance on three levels:

Track record. Last December 69% of the first 848 first members of the Panel said that they would like David Cameron to be the next Tory leader and 31% said that they hoped David Davis would become Tory leader.  The actual result was 68% to 32%.  This group of 848 people continue to be the bedrock of the Panel.  Nearly all continue to participate on a monthly basis and I believe that they are a good guide to Tory opinion.

Weighting.  We compare the first question of the poll (which is always the same) in the previous and current surveys and take as our reading the actual change within that group (the matched group is about 95% of all respondents). We also match current respondents to the original 848 respondents as a control. Having weighted the first question, we then weight all subsequent questions to that first question. So far, weighting has never required more than a 1% adjustment, as the panel is in fact very stable, with new respondents closely resembling the profile of the earlier respondents. (Supporters of neither candidate in last year’s leadership race are proving to be disproportionately slack or enthusiastic at participating.)

Recruitment.  My reason for believing that the Panel members represent the membership is the routes via which we have been recruiting members.  The Cameron and Davis campaigns both emailed their supporters in order to alert them to the existence of the survey, and party organisations like the Conservative Christian Fellowship, Conservative Way Forward, Conservative Friends of Israel, Tory Reform Group and Women2Win were invited to alert their supporters to the surveys.  Since then I have emailed Tory Associations and leafleted Conservative events.  As I leafleted the Spring Forum in Manchester it was striking how many of the people I invited to join the Panel said that they were already signed-up. One possible criticism which I do accept is that the Panel represents activists rather than the full range of the non-activist membership. However, there is no evidence that non-activist (or non-participatory) members are essentially different in their views from activists. Any evidence about this is virtually impossible to obtain. It would therefore be wrong, and indeed meaningless, to attempt to draw any conclusions about the views of non-participatory members.

I do not pretend that the online survey of the kind being run by ConservativeHome is perfect.  There are probably some people who vote in the surveys who are not paid-up Tory members and there may be some infiltrators from other parties.  But the three reassurances above will, I hope, offer readers of the survey a sense of the Panel’s overall reliability.  Over time I hope to add one or two more security measures.

While we are on the subject of the Panel it might also be helpful for me to restate why ConservativeHome runs it at all.  The survey results don’t always make comfortable reading for the Tory leadership or for individual shadow cabinet members.  Some of the visitors to this site who are most loyal to the current party leadership believe that the survey provides ammunition to the party’s opponents and I am not insensitive to their concerns.  I, for example, delayed publication of the April dip in satisfaction with David Cameron because I did not want to encourage other parties’ council candidates on the eve of May 4th’s local elections.

The reason why ConservativeHome will continue to survey the Tory grassroots/ webroots, however, is our fundamental belief that the party is not owned by the current leadership or any leadership. Gone are the days when CCO can purport to speak on behalf of members to the press without fear of contradiction.  ConservativeHome believes that the leadership deserves loyalty and respect from the grassroots but loyalty and respect is a two-way thing.  The party must listen to the grassroots.  We are the people who raise the party’s funds and deliver the leaflets.  ConservativeHome’s first campaign was the successful championing of the membership's right to a vote in the leadership election and, much more recently, the membership’s right to be able to inspect the candidates’ A-list from which the next generation of Tory MPs will be drawn. ConservativeHome’s mission involves helping to build a more open, democratic and participative Conservative Party.  The voice that the Members’ Panel gives to the grassroots is a fundamental part of that mission.

***
Recent Panel findings: David Davis tops satisfaction league table, state funding of political parties rejected by 81% of Conservative members and 54% of Tory members say the war in Iraq was a mistake.

Comments

[Wat Tyler]

Go Tim

ConservativeHome has become a national institution. As a counterblast to the in-built centralising tendencies of senior management it's value is well beyond rubies.

Of course, the surveys may not be as scientific as the mainstream polls like to claim they are. But they seem to me to capture precisely what most grassroots Tories are saying round our way.

Keep it up.

[Editor]

Thank you Wat.

One funny story about the Panel...

I believe that a good number of Tory MPs are Panel members and, just possibly, a few of the ten senior shadow cabinet ministers we regularly rate. One of those ten recently joked to me that he/she doesn't vote so much as to boost his/her own rating but to vote down the rivals!

My lips are sealed as to the identity of that individual!

[James Morris]

Tim

Great Stuff. It would be good though if we could develop mechanisms for getting opinions on the Conservative Party across the Conservative coalition in the broadest sense. As we have discussed before the nature of party political affiliation has changed and is changing and though your survey of members is incredibly valuable it would be good to find ways of connecting with opinion outside the clearly defined boundaries of membership.

[Henry Edward-Bancroft]

Wouldn't it be great if we could back up the excellent work done by ConservativeHome with greater fundraising efforts? Now let me think... ;)

[matt wright]

While there are some people who post on this site who are just negative or anti DC, I have to say that on balance this site is a great benefit to conservatism. The panel is generally representative of party members and it shows the very strong support DC has. I know this is reflective of the position amongst members as when I attend association events, a large majority are strongly behind DC but there are a few loud sceptics. In some ways the ability of my party to engage with this modern form of debate is a test of its wish to be more local. I like the way that recently Francis Maude has posted to the site and got involved. In any party or amongst its supporters there are different views, that is just life. I face the same amongst my constituents as a councillor or colleagues at work. What sensible people do is listen to all this and in some cases change their approach or in other cases reflect on this and explain why they will not. Change is not a fixed plan it is about growth,

Matt

[Zhukov]

I think a lot of the comments, certainly mine, on Guido's site were less about the surveys and more about the comments being left on the site.

Although Conservativehome is a fantastic resource for news the comments section has become a forum for oddities like William and Chad, people with their own narrow agendas.

[Chad]

for oddities like William and Chad

LOL

* Against state-funding of political parties
* Against positive discrimination
* pro-grammar schools
* pro EU referendum.
* pro low tax environment

Such beliefs used to be called 'conservative' but if they are now called 'oddity' then I am very happy with that label!

[Henry Edward-Bancroft]

If I did have a criticism it would be that shared by "Zhukov" the discussion does tend to be monopolised by a handful of people with entrenched views.

While I wouldn't go as far as to call them "oddities". If there was some way of broadening out the discussion that would be helpful and, i think, aid the quality of debate.

[Chad]

Henry,
Would you care to provide an example of one of my 'entrenched' ideas that do not stand up to rational criticism?

The last time I checked, 81% of conHome members agreed with my opposition to state funding of political parties.

I would suggest that it is the party leadership who have entrenched views.

By 'quality' I guess you mean agreeing wiht Cameron.

[Esbonio]

Well said, Tim.

Whilst not a regular poster on your site, I have been reading it and making occasional comments from its inception. I find your site excellent in all respects and believe you provide an invaluable service. Your aims and your methodology seem fine to me. Please continue the good work which includes allowing a variety of opinions to be expressed.

[Henry Edward-Bancroft]

Chad,

I certainly don't disagree with much of what you have to say and often quite enjoy reading your posts.

[James Burdett]

I find this site interesing but agree with Zhukov up to a point. There do seem to be a heck of a lot of indivisuals on here who want to occupy certain idealogical ground regardless of how broadly attractive it is. Many seem to hold a 'mountain must come to Muhammed' tendency and not see the postive virtue with engaging with the electorate where they are not where you would like them. It is particularly depressing that many seem to be the younger posters, the older hands seem at least to see the virtue of pragmatism.

[Editor]

Chad et al: I'd be grateful if this thread didn't become a debate about the qualities of the various people who leave comments on this site.