I’m not a natural Tory (if you’ve met me, you’ll know that I’m quite the opposite) but I can’t help but be impressed with their grasp of a few of the opportunities offered by new (potentially) democratic tools lately.

The first one is their use of Google Moderator in the Q&A that is embedded in their draft NHS manifesto launch. It’s a very savvy way of avoiding the appearance of control-freakery that has dogged Labour’s ‘Big Conversation’ while at the same time weeding out the trollery that neuters a lot of online political discussions.

But on a bolder canvas, their offer of £1m to the developer of the website that can “harness the wisdom of the crowd” by producing an online platform to solve “common problems” is very striking. The dovetailing of this approach with Cameron’s notion of the post bureaucratic age is very deft.

There are a number of constructive responses to this, but they don’t really include Marina Hyde’s bit of hired trollery. Rosa Prince at the Telegraph has picked up on what appears to be an imaginary hostage to fortune here – the opportunity that the internet offers to give the people who call Radio 5′s 606 phone in the opportunity to pick the England team. The Ebbsfleet United project was designed to offer living proof of this blog’s central thesis – the value of representative democracy – and Anthony’s account of it here is worth revisiting this. But as far as I can see, Hunt has simply not mentioned the idea that the England squad is a suitable candidate for crowdsourcing.

In a recent Times column, Matthew Parris touches on a fair bit of what must – and should – make up the Tories motivation in all of this:

Responding to Margaret Thatcher’s unfairly quoted remark, “There is no such thing as society . . .”, Mr Cameron has said, repeatedly: “There is such a thing as society; it’s just not the same as the State.” I’m sure Cameron wants the phrase to be seen as a philosophical anchor.

He concludes:

There will never be a better time to make that journey, now everybody knows that the State cannot afford its present level of beneficence. Do Cameron Conservatives acknowledge, even to themselves, that if they are going anywhere at all, this must be their destination?

There is a sweet spot here: where doing the right thing is also doing the populist thing and doing the Tory thing: a move to break the monopolies that dominate policymaking – the think-tanks, pressure groups, civil servants and party bureaucrats – is long overdue. Only £1m to fix such an obvious glaring hole? As a Labour supporter, I find the neatness of it to be quite crushing.

But beneath that, a serious attempt to gamechange the failing public policy processes  needs to understand what causes those failings. Using the internet to break one stranglehold only to step into another one would be disastrous, and the Tories do have bad form in this department. For example, this approach needs to be a good deal more literate than last years’ bloody awful ‘localism’ proposals.

What stands out from the Editors Notes in the press release is that – largely – this plan is avoiding many of the populist forms of crowdsourcing that are likely to result in poor-quality policymaking. The refreshing thing here is that there is no simplistic appeal to anyone with a bit of time on their hands to Have Your Say. This is an important concession. There is, however, a danger lurking in the final bit of the editors notes – ‘Harnessing the wisdom of crowds in policymaking’. It looks suspiciously like a muffled appeal for a lot of expressed opinion.

I have a sneaking suspicion that most of the editors notes were written by … ahem …. Tom Steinberg (who does know what he’s talking about and has the imagination to adapt good ideas from elsewhere) and then that last one was added by a politician who didn’t really understand the idea properly.

There are a number of reasons why the harvesting of expressed opinion will not necessarily (or even usually) result in good quality policy-making. These are – in no particular order:

The ‘detached wisdom’ problem: if you want to find out what people really think, you have to find a way of getting a representative cross-section of people to lodge their mild preferences. Stockbrokers will often advise you make your best judgments when you are not emotional about stock. There appears to be a parallel here with The Long Tail argument: That the most evident views on any subject are either those who really REALLY care or who have a vested interest in a particular outcome. The larger volume, however, comes from the general bubble of conversation that is largely ignored by the media and by politicians. If policymakers only have the opportunity to hear what self-styled experts or enthusiasts have to say about a particular subject, they will miss precisely the wisdom that Jeremy Hunt seems to be looking to harvest.

The ‘active citizen’ problem: related to the ‘detached wisdom’ one: Time-rich, often more materially wealthy than average with particularly strong views on specific issues. It makes for shrill, populist, non-inclusive policy-making and it foregrounds the concerns of a small social group over a wider one.

The ‘convening power’ problem: that wealthy or influential individuals (newspaper owners, celebrities) can campaign on particular hobby-horses at the expense of the issues that have more general lightly-held support from a wider range of people.

The groupthink problem: There is a point here that all points of the political compass should really acknowledge: That high-quality thinking is incompatible with the kind of groupthink that mass-media led politics promotes (a central plank of James Surowiecki’s Wisdom of Crowds thesis, by the way). This one is related to the ‘convening power‘ problem. Processes designed to harvest observations from the public will be gamed by political opponents, and particularly by newspaper editors. Outliers will be castigated and apostates will be hung out to dry. It’s also too good an opportunity for your political opponents to turn down.

I’ve argued before that the best way to harvest The Wisdom of Crowds is to quietly eavesdrop upon it rather than do it in a high-profile way. The form that questions take is important as well – asking the public to describe the problem rather than shout their proposed solutions at you. And these are the tasks that the semantic web can solve. It offers a tremendous potential for new forms of collaborative working and interdisciplinary exchanges. Again, the Tories are hitting exactly the right notes by promising to free up public data.

At this point of this post, I need to link to a good easy-to-read politician-friendly article entitled ‘How the semantic web can crowdsource high-quality judgment and improve policymaking.’ I can’t find any at the moment, but if anyone has seen one, I’d welcome the link.

By the way, I’m not completely opposed to the idea of harnessing expressed opinion and finding ways of getting value out of it. This guy seems to be looking for this idea and I hope that they talk it through. But if the Tories want to spend their money wisely, they need to work with the sort of people who could write that essay and build a spec around it.

In fact, here is an idea that I’ll give them – something that they can do before they get their hands on that £1m Cabinet Office nest-egg: They can take the Conservative Party’s petty cash tin out and offer a £1,000 prize for the best essay entitled ‘How the semantic web can crowdsource high-quality judgment and improve policymaking.’

It would need to be written for an audience of politicians and party bureaucrats and it would need to soft-pedal on the threat that this approach holds for party bureaucrats as well….

I saw this press release yesterday and it’s had a bit of take-up from the press. Sadly, none of the coverage that I’ve seen has addressed what seems to me to be a serious and interesting – though problematic – proposal.

Jeremy Hunt MP

I’ll be writing something myself over the next few days, but in the meantime, here is the actual press release in full.

The most interesting bit – the ‘notes to editors’ – hasn’t been published online anywhere as far as I know and it’s really worth a look.

I’d be interested to see what you make of it:

Hunt: Solving problems together: Harnessing the Wisdom of Crowds

The Conservatives are today announcing a competition, with a £1million prize, for the best new technology platform that helps people come together to solve the problems that matter to them – whether that’s tackling government waste, designing a local planning strategy, finding the best school or avoiding roadworks.

This online platform will then be used by a future Conservative government to throw open the policy making process to the public, and harness the wisdom of the crowd so that the public can collaborate to improve government policy. For example, a Conservative government would publish all government Green Papers on this platform, so that everyone can have their say on government policies, and feed in their ideas to make them better.

This is in addition to our existing radical commitment to introduce a Public Reading Stage for legislation so that the public can comment on draft bills, and highlight drafting errors or potential improvements.

Launching the competition, Shadow Culture Secretary Jeremy Hunt said:

“Conservatives believe that the collective wisdom of the British people is much greater than that of a bunch of politicians or so-called experts. And new technology now allows us to harness that wisdom like never before. So at this time of year, when families and friends are getting together, we’re announcing a new idea to help the British people get together to help solve the problems that matter to them.

“There are currently no technological platforms that enable in-depth online collaboration on the scale required by Government – this prize is a good and cost-effective way of getting one.

“Too often policy has been ill thought through with disastrous consequences. When formulating and implementing policy why should we not listen to the hundreds of thousands of experts out there?”

ENDS

Notes to Editors

In the bureaucratic age, decisions in government, business and other organisations were typically made by a small, closed group of experts. In the post-bureaucratic age, new technologies enable us to reject this top-down approach to decision-making. These new technologies allow us to harness the wisdom of the crowd, take advantage of the power of mass collaboration and make use of the information and ideas dispersed amongst large groups of people. Evidence from around the world has shown that this post-bureaucratic approach can result in more efficient and effective decision-making and problem solving than relying on small groups of experts.

Harnessing the wisdom of the crowd in this way is a fundamentally Conservative approach, based on the insight that using dispersed information, such as that contained within a market, often leads to better outcomes than centralised and closed systems.

The Conservative Party has already used crowd sourcing to develop new policies, for example through our ‘Stand Up Speak Up’ initiative. To make sure that we make best use of this approach, a Conservative government will offer an unprecedented £1 million prize for any individual or team that develops a platform that enables large groups of people to come together online to solve common problems and develop new policies.

Harnessing the wisdom of crowds – case studies

Innocentive
Innocentive is a website used by leading companies such as Proctor and Gamble and charities such as the Rockefeller Foundation, to tap into the wisdom of the crowd and get answers to otherwise intractable research problems. There are over 160,000 scientists and other experts in the Innocentive network, and they are incentivised to take part through cash prizes for solving problems.

Improving the Netflix algorithm
Netflix, a US-based DVD rental company, wanted to improve the algorithm it uses to recommend films to users. Instead of hiring a research team itself, it threw open its dataset, and offered a $1m prize for anyone who could improve its algorithm by 10% or more. This approach yielded a solution far more cheaply and quickly than relying on an internal team of researchers.

Peer-to-patent
Peer-to-Patent uses the wisdom of the crowd to improve the patent process, and has been trialled by the US Patent Office. Under this approach, patent applications are posted online, so that instead of relying on a small group of bureaucrats, anyone in the world can check whether the application is valid. This approach seems to be much faster and more efficient than the traditional closed approach to appraising patent applications.

Solving maths problems
In January 2009, Timothy Gowers, professor of mathematics at Cambridge University and a holder of the Fields Medal, posted a hitherto intractable maths challenge on his blog, and invited readers from across the world to collaborate and solve the problem. The resulting comment thread spanned hundreds of thousands of words and drew in dozens of contributors. Six weeks later, the theorem was proved.

Harnessing the wisdom of crowds in policy making
In the post-bureaucratic age, opening up the policy making process can help us to design better policy and transfer more control to individuals and communities. The Conservative Party is committed to harnessing the wisdom of crowds in a number of ways:

• We will introduce a Public Reading Stage for legislation, so that the public can help to spot errors in legislation, and feed in their comments during the legislative process.
• We will set government data free, enabling the public to collaborate and develop new social and commercial applications.
• We are publishing online, and in real time, the expense claims of our Shadow Cabinet, enabling full and instant scrutiny.
• We have published online a leaked version of the Government’s IT strategy, so that people can post their suggestions on how to develop a better set of policies.

A Conservative government would seek to make extensive use of this approach. However, there are currently no technological platforms that enable in-depth online collaboration on the scale required by government.

We are today announcing that a Conservative government will offer a £1 million prize for any individual or team that develops an online platform that enables large scale collaboration and meets the specifications that we will be publishing alongside the official opening of the competition following the election. This platform will then be used by a future Conservative government to throw open the policy making process to the public, and harness the wisdom of the crowd. For example, a Conservative government will publish all government Green Papers on this innovative and open platform.

The source code of the platform will be made openly available, so that it can be used by local councils, social enterprises and other organisations free of charge.

While leading institutions such as the Gates Foundation, Google and Netflix have successfully made use of procurement prizes, this £1 million prize will be the largest prize ever offered by a British government in the modern era.

(With thanks to Craig Elder for supplying the original)

Bless me Father, I've been a bit certain about something...

In a very good edition of BBC Radio 4′s ‘Analysis’ programme towards the end of last year, the columnist David Aaronovich recounted a programme that he produced in the 1980s featuring the Archbishop of York, John Hapgood.

The Archbishop, as far as I can see, had the kind of views that would appeal to a Guardian reader rather that an Anglican traditionalist.

Jonathan Dimbleby asked him if it wasn’t the case that people needed a bit of certainty about big issues in order to live their lives. the response that the Archbishop gave stunned Dimbleby and Aaronovich. He said…

Has it occurred to you that the lust for certainty may be a sin?

The whole programme is really worth listening to – I think that podcast subscribers get the option to download all of the archives and the transcript is here.

One of my favourite political bloggers, Chris Dillow of Stumbling and Mumbling has written a great deal about the curse that the apparent need for certainty places upon democratic politics.

Just for reference, all of these posts are worth reading, but Chris’s strapline – ‘an extremist, not a fanatic’ is probably traceable to his previous career as a stockbroker – and the advice that every trader receives during their career – that not being sentimental about stock is a good thing – and that fanaticism always clouds judgments.

Chris often promotes a Rortean irony as a way of viewing the world, and seems very stuck by James Surowiecki’s Wisdom of Crowds thesis – an attractive one, particularly to those who are more open to ideas of direct democracy than I would be.

And what does this distrust for certainty mean for advocates of local democracy? I’d say that it tells us that a great many consultations throw up the most useless information, as opposed to the most useful. If the general public are widely seen as being too apathetic to turn up to a polling station every few years, the idea that the bulk of people with lightly-held preferences will participate readily on a subject that they are not too bothered about, is a bit outlandish.

So we have, instead, the usual suspect problem. Where people with views that they hold fanatically are very keen to participate, and keen to be heard over the noise of the general public. People who have a vested interest are also likely to be much in evidence. But the eavesdroppable conversation – the one where ordinary people who hold their views fairly lightly meet – is one that is never fostered by most consultations.

As a result, we end up with the kind of balance that pervades so much of public life – one where balance is equated as being the mid-point between two poles of groupthink – rather than the balance that emerges from a wide range of views.

Which brings us back to the parliamentarian argument – the need for the distributed moral wisdom of the elected.