Thanks again for all of the feedback on those open data posts recently.

Just to recap, I’m helping to organise an open data project for some school pupils within the a London borough in the new year. One of the big tasks is to flush out all of the data that may be available.

I’m going to be taking subject areas such as crime, health, education/children’s services separately and posting on each of them, using the  links and a few ideas that have come from different directions.

My first subject, though, will be on voluntary/civil society activity in a particular borough – in this case, Barnet.

This is a good example of a data-set that isn’t generally available yet in any standardised form, but one that may be of interest to school pupils in mapping some aspects of their locality.

In terms of drawing down experience of a local voluntary sector and open data, Jo Ivens in Brighton has pointed me to the Data for Neighbourhoods and Regeneration site here – a very good set of signposts – along with her own Databridge site.

I started to try and summarise a few good points from this site but ended up finding all of it worth reading – it will prove to be an incredibly useful resource for everybody involved in this schools project. As a taster, I’m shamelessly pinching this video, but the whole site is worth a visit.

I’ve also had a helpful conversation with Ruth Mulandi, CEO of Community Barnet – the local voluntary sector hub – and this is what I found out.

Firstly, they hold most of the data that may be useful within their website Content Management System (the tool that they use to maintain their website).

This includes their directory of community organisations called InBarnet. They have over 1,000 community organisations registered within the borough – around 850 of which are active.

Being able to download this would be very useful, and I’m told that it’s possible, subject to a few caveats:

• The database includes information that individual groups have submitted to CommunityBarnet, but of it is on a ‘not for publication’ basis (in some cases individual phone numbers, contact details etc) and it is subject to some data protection rules
• CommunityBarnet don’t have the resources to regularly run bespoke dumps from this database at no cost– they have one person managing all of this and it’s not a full-time job by any means so additional data work needs to be resourced somehow
• It is an ongoing project to get all of the info about all of the groups that they ideally want to provide, such as what each group does, where, when, what type of service they provide and how etc etc, and to keep this up-to-date for all of the 800-odd active groups on the borough
• The database on the website is searchable, but obviously not all of the data is there (if groups have not provided it yet)

However, leaving aside some of the data that cannot be fully shared with third parties their website gives the undertaking that…

We can provide more specific reports , including:

Type of service provided: one to one support, counselling, after school clubs, befriending, advocacy, day care services, mentoring, training, and many more groups and individual needs served: children, adults, carers, parents, mental health, learning difficulties, cultural and faith specific and many more

Some of these groups provide specific services (lunch clubs, advice, day-centres, etc).

My contact at Community Barnet has offered me any reasonable amount of help in extracting this information in a useable form, but I’ll have to bear in mind the constraints that they are under in doing this.

So, what else is there?

In this document, we see some results from a 2009 survey in which 25% of the active organisations  on the database provided a response outlining what they do, who they do it for. Again, Ruth tells me that this is all held within that unified database held within the website management system, so it should be very simple.

I suppose my big question is this: How long before some government agency starts to standardise the collection of data about voluntary sector activity for publication? Mapping these services would surely have some use – not least to the local authorities concerned.

With the ‘Big Society’ as such a priority for the current government, it can’t be too far off, can it?

A while ago, I posted here giving reasons why I thought it would be a good idea to start involving school pupils in the processing of public data.

There are strong democratic arguments for doing this – ones that aren’t immediately obvious. There are also good ‘transparency’ arguments (but I’d make my usual point here about transparency and democracy not always pulling in the same direction).

Local Ward Atlas data - click to explore it

There are two other reasons why this is worth doing:

1. It’ll be fun to do. School pupils, doing all kinds of things with data that their older neighbours wouldn’t value just for the hell of it. Anyone watching this will learn a lot and probably have a laugh while doing it
2. It will be a good thought experiment for everyone involved. In my experience, most people who work in or with local authorities don’t really understand the potential to do good things here.

I’ve never seen anyone try to pull together a good index of all of the relevant and interesting data that is available within one local authority area with the aim of giving school pupils something to work with, so over the next few weeks, I’ll be doing exactly that.

In this case, I’ll be looking at what data we can find on the area covered by the London Borough of Barnet (I live there, and the council have expressed an interest in this anyway) from a variety of different sources.

I’ll be writing a short article here on each of them outlining what they have and how it could be used, and hopefully sharing a few of them on the London Data Store blog. I should add here that a lot of what follows has resulted from conversations with friends, too numerous to credit here, but I was give a good initial steer by Emer Coleman at the London Data Store who has a strong local authority background.

I’d really welcome your feedback on any of this.

So, my first question; Are there any obvious omissions from this list of sources (below) that I’m going to go to for data that we can use with school pupils at a data-hack event?

• The council themselves – for demographics, expenditure, service provision and take-up, revenue and other relevant data. There is currently a Maps Facts & Figures page on their site, but I think that there could be more ‘machine readable’ data that we could get from them with a bit of help
• The data behind the police crime-mapping services
• The London Data store - loads of information from a wide variety of different subject areas
• The local Primary Care Trust /NHS
• The local voluntary service council

There’s one further area that has been suggested to me. School pupils are likely to be very interested in Children’s issues anyway, and every local authority commissions some research that doesn’t fit into national frameworks. So I’m going to be having a conversation with the Children’s Services office if I get the chance. In addition, any information I can get on schools will be particularly useful for the same reasons.

If my own children are anything to go by, I suspect that they will want to move quickly beyond the data that we provide them with and start creating their own information. There’s a huge wealth of information that children could provide about their local area – data that could be crowd-sourced with a bit of creative thinking.

We will need to ask them – or even encourage them to do the asking. This is, of course, the holy grail of democratic data-use – participation and co-design. But for now, I’d like to explore the limits of the data that adults have provided. At the moment, many adults don’t really understand that a huge variety of data-types + analysis can be very valuable.

We can walk now. Running comes later.

It’s late on Friday afternoon – here’s some brain-candy to chew on over the weekend.

Here’s Douglas Rushkoff – one of the most established commentators on interactive communcations explaining the cost of transparency. It’s liberating stuff – yet a lot of it seems so straightforward in Rushkoff’s hands. It often reads like the bleedin’ obvious. A lot of it is aimed at the individual, discussing their rights and the way they are manipulated and exploited.

Douglas Rushkoff: The Future of Transparency from Applied Brilliance on Vimeo.

There’s not much in here that seems directly aimed at a local government audience (indeed, nothing expressly) yet I’d suggest that it’s hugely important to grasp the power-relations that effect us all – and Rushkoff is great for that.

One possible lesson though: how important it is to engage all council employees more in engaging with local people.

I spent Saturday at Local Government Camp in Birmingham – there’ll be at least one post along here shortly based on things I learned there. But this one is here to host the slides I used at the start of the conversation (sorry – Slideshare is being a complete pain today and I can’t embed the slides for some reason):

Along with @tobyblume, I initiated one that was intended to be on data visualisation and how schools could be more effective partners in this. It was based on the idea posted here the other day.

We kicked off the session identifying what visualisation is intended to achieve – at least in democratic terms (the slides – above – are based on this post that I wrote for my business blog) – and in the end we didn’t get much beyond this issue and it’s implications for local government’s corporate culture, but I think that the observations that came out of it were very useful indeed.

The conclusions the group reached can be seen on the final slide. I’d be interested to know if you think that there are any obvious lessons that we missed?

It’s a regular theme of this blog that transparency and open data – while undoubtedly being good things – can often create situations in which democracy is diminished rather than enhanced.

The other day, for example, I posted my misgivings about guerilla webcasting of council meetings. (Shorter version: can result in selective reporting, poorer press coverage and increased power for small heavy-preference pressure groups – boo!)

Looking at it from the point of view of a local authority (particularly the communications team as well as the councillors) transparency and open data seem to have created a situation where the amount of time spent dealing with the angriest local residents goes up.

That the armchair auditors – far from being constructive partners – are non-neutral political activists [this post makes this case in more detail] who are selectively disrupting the aspects of the local authority’s work that they don’t like.

And this can be a good thing – up to a point – but it fundamentally undermines the duty of a democracy to address the concerns of the whole electorate – and not just those with time on their hands.

You could say that Councils only have themselves to blame for this. If information has to be dragged out of them by FOI requests, then only people with time on their hands will do it. If information isn’t attractive to engage with, then only people who have the time to puzzle it through will do it. [Again, this argument is made in more detail here].

If information isn’t made available in a format that allows other websites and forums to consume it, then the only people who will look at it are the lean-forward activists who can trawl the Council’s website. Good, attractive, easy-to-read, well-presented information can – and will – be linked to from Facebook groups, Mumsnet, Netmums, hyperlocal sites and other relevant local forums.

So this brings me to the suggestion: How can we get a lot more of the key information that Local Authorities provide about themselves into an easy-to-understand, easy-to-engage-with, easy-to-share and easy-to-mashup format quickly and easily?

Here’s how. A local authority could engage with the Design/Technology teachers in their local authority area. They could show all of the useful ways that council data can be presented – from the high-end where data is extracted, cleaned-up and poured into a visualisation tool (this Redbridge local business mapping project is very much the poster-child for this kind of thing as far as I can see).

In addition, they could show good examples of how short YouTubed videos, presentations (slideshared or using Prezi, for example) can make more engaging presentations than raw CSV files or PDFs posted on the council website. It doesn’t even need to be that sophisticated.

Every page on a local authority webste could probably be re-written and presented in a more easy-to-engage with way by a user of the service than by a provider. I mean no disrespect to local government Comms people in saying that their audience can probably explain Council services better than the providers of it can – once that audience understands what the services are.

So how about this? Councils can hold a borough-wide competition – facilitated by local schools – for the pupil (or group of pupils) that takes some information that the local authority is willing/obliged to provide and creates a consumable, re-usable artefact that makes it easier and more attractive for the public to understand and enage with the information.

Talking to a few teachers I know, there are a number of ways that this can fit into the curriculum both in terms of design/technology and citizenship.

And if this is a good idea, then I think local authorities will need the following resources to draw upon:

• An outline of which parts of the curriculum this can enhance – essentially something that helps councils sell the idea to local teachers
• A good set of web-pages to look at that have how-to videos/worksheets/presentations/infographics
• A set of suggestions that councils can give to teachers (e.g. “why not get data about which businesses we deal with and put it on a map to encourage is to source more local suppliers?”)
• Guidance and contacts on how to run a local hack-day (it would be reasonable for a council to pay a few good data-visualisation people to facilitate something like this) to bring expertise into the process
• Advice on the practicalities (CRB checks, how do find a venue and organise a hack-day)
• Suggestions on how the motivation/judging can be done and how they can generally nudge-up the quality of the work provided by the schools and their pupils

I’d be interested in seeing if a few people would be interested in pulling together a set of re-usable resources like this that can help councils do borough-wide schools open-data projects. It will be good for the quality of education and – I beleive – result in a broader and more conversational engagement with local citizens.

Is this a suitable idea for an event like Local Gov Camp on Saturday perhaps?

The news that a blogger who filmed a meeting of a local council in Carmarthenshire was arrested for “breaching the peace” raises an interesting question that could have a slightly unfashionable answer.

Flip-cams: The sword of transperency or an engine for selective reporting?

My friend, David Allen Green, writing in the New Statesman has a supplied a detailed trawl of the legal evidence along with some good journalistic legwork to conclude that…

“…Carmarthenshire Council and Dyfed Powys Police have simply acted in an altogether hapless, illiberal, and alarming manner. A person, surely, should not be arrested and detained just for filming a public council meeting, and a council should not be able to prevent someone from doing so in this manner. In my opinion, all the councillors, officials, and police officers involved in this sad sequence of events really should be ashamed of themselves.”

I’m in two minds over whether the flip-cam will improve the quality of local democracy, and I think this highlights some of the tension between liberalism and the good practice of liberal democracy.

On the one hand, all of David’s arguments stand – and then some. Surely a good democracy should remove any barriers that stop people from viewing democratic proceedings? Transparency will result in less corruption, better decisions, a greater sense of participation, and so on.

On the other hand, there are a few snags. Firstly, if individuals are doing this selectively – on issues that they care about, we will naturally expect to see a bias towards issues that small groups of individuals care about. These may be (but not always will be) subjects that effect people who have more time/resources on their hands. If this becomes the main way that Council meetings are covered, it can expose councils to more pressure group politics.

This may be at the expense of the decisions that many of us – people with mild preferences and a need to see policy serve the interests of the whole community – would expect to see from local authorities. I know I trot it out a lot, but the example of a more direct democracy provided by California should be a shocking example to us all.

So, is a balanced view of proceedings going to be distributed by citizen journalists? Or will it inevitably result in selective reporting?

Blogger arrested for filming council meeting http://t.co/4CpUfyN MP and public barred http://t.co/wwwR6Gs who do these ppl think they are?

June 12, 2011 12:16 pm via Twitter for iPhoneReplyRetweetFavorite

@bengoldacre

ben goldacre

Another question: Does this drive up the quality of deliberation (i.e. do Councillors raise their game?) or does it result in either a more guarded approach or, conversely, a more soapboxy style?

We expect councillors to behave in a disinterested way and there must be some parallel here with the jury room: Would juries make better decisions if they were selectively recorded?

Then there’s the question of professional journalism. I know this is a moot point at the moment, given the headlong retreat from half-decent reporting among local newspapers. But we expect pro-journos to cover issues in an even-handed way, catering to mild preferences of a broad audience rather than the narrow views of a deeply interested one.

Will this kind of guerilla coverage drive this kind of reporting out? I posted here a couple of years ago about the negative impact that opening Parliamentary proceedings out had – the conclusion was that newspapers stopped covering it because anyone who was really interested could get the real thing on the radio or (later) TV. Will this apply to local authorities, or are we in a different ballgame with local politics?

In my experience, the answer may be that councils should routinely film all proceedings and index them professionally. This will reduce the scope for selective coverage and allow visitors who follow the links the option to hear all of the points (and the context provided by professional local government officers) to the proceedings. It will also allow people to drop randomly into a council meeting and see a broad range of issues being discussed responsibly (or not).

A firm in Brighton called Public i (declared interest: I’m personally friendly with a few of the team there) have been offering a web-casting service aimed at local authorities for some years – it indexes each speaker which saves you ploughing through whole meetings if you’re only there for a particular reason [random example here].

Surely councils need to pro-actively promote a public awareness of the whole of their work? Given the low interest in local politics, it will make it easier then for local journalists to report procedings and may result in more broad coverage. Unless they do this, we can expect selective reporting to dominate agendas more and more.

The Rt Hon. David Laws MP. Click pic for credit

There was an interesting review of a study around politically connected firms on the BBC’s Thinking Allowed programme recently asking how far different countries find their governance effected by the relationships politicians have with previous (or current!) employers.

The early coalition-casualty, David Laws – for example – is a former Vice President of JP Morgan and Alan Duncan used to work for Royal Dutch Shell.

The programme quotes a recent article from Mara Faccio, a US-based Italian economist whose work on politically-connected firms (lots of links from her homepage here) is, in part, er…. inspired … by a desire to understand Silvio Berlusconi’s grip on the Italian state.

Unsurprisingly, there’s a direct correlation between press freedom and low levels of corruption. In the UK, 150 firms have connections to MPs – the highest level of interpenetration anywhere in the world. It’s quite a surprising statistic initially, but in the UK, politicians are offered opportunities to join boards rather than the other way around – where businesses put their people into politics (the strong contrast with Italy). As a result, the benefits to the firms were, according to Faccio, negligible.

It does put all of the recent expenses scandal into perspective though, doesn’t it?

Listen to the whole thing though if you can?

Anyone who follows the BBC News site, or who reads a newspaper, will be familiar with a good few interest groups and think tanks. Where their news releases aren’t the entire basis for the story, they are invited to comment at length, in the name of political “balance”, or on the basis of an often-undeserved authority.

The TPA: Corporate shills?

A great deal of our time as bloggers is thus spent exposing the same old partisan front groups – the left are interested in the TaxPayer’s Alliance, – corporate shills, and organisations that exist purely and simply for the promotion of a particular set of views. Right-wing bloggers hunted down a significant scalp last year, taking out the earlier incarnation of Labour List – a site that appeared to simply be a political attack dog, and one that wasn’t embedded in the better instincts of the blogsophere.

While individuals can always change their mind on an issue, interest groups cannot, and will not. Moreover, their neatly packaged set of proposals can be tempting for governments running short of ideas, and short of friends.

Whether the groups are ostensibly on the left or the right, their influence can only be bad for politics. In the name of “balance”, the essential politics within a debate – the key issues and arguments – is drained away, or rechannelled to the financial benefit of one organisation.

Why should we – as bloggers – put up with seeing the same discredited arguments trotted-out again and again, and which we have argued over and over again, when we have – at our disposal – a vast resource of evidence, argument, and opinion to call upon?

What I propose is a collective – and non-partisan – organisation of political bloggers, which will challenge the interest groups in the name of honest and open politics, and provide newspapers and online news sites with a central resource that allows them to dispense with the services of self-interested think tanks and self-styled experts.

These bloggers would come together whenever required, putting aside their partisan interests, to tap the blogosphere’s collected wisdom – and, just as importantly, its memory – to fisk, rebut, and generally trump groups that are used to thinking in relative isolation. Bloggers will have many advantages, not least access to background information about the groups, via SourceWatch et al.

Of course I don’t propose that bloggers attempt to produce consensus. Not only is this implausible, but it would be dishonest, and would drop us into the same trap as the BBC and others. While artificial disputes help no one, genuine disagreements must come out. The public must have access to the full range of political opinion – no political cause benefits in the long run from anything different.

All in all, by shaking up lazy journalism, exposing lazy thinking, and by undercutting the “go-to” groups and “experts”, the reputation of the political blogosphere – as simply the best resource for distributed thought and opinion we have – must surely increase.

Please join the Positive Political Blogging campaign on Facebook, and help spread the word!

Dave Briggs has a good post up about how organisations introduce technology. He contrasts the ‘JFDI’ approach (which stands for Just Do It) and a more boring sustainable approach.

Tactics: For teams that are scared of losing.

I’ve met Dave and he has very sensible views on Football. Our mutual friend Brian Clough could have contributed to this whole discussion. As he put it (here) …

“…tactics aren’t for me, they’re things that teams dream up because they might lose”

Translating that into Dave’s question, it’s basically about getting organisations to increase their desire and competence in the field of ‘interactivity.’ The rest – the tactical choices of which technology, the timelines and roll-out plan – become somewhat redundant when you are able to deploy a team that know what their jobs are and are happy, competent and confident that they can do them. This question applies to local authorities a good deal.

On the day that David Cameron is majoring on his notion of The Post Bureaucratic Age, there is a lesson here: So much of the division between the boring ‘tactical’ practitioners and the JFDI enthusiasts stems not from a gulf in the understanding of the technology, but in the willingness of individuals within organisations to interact.

I’ve spoken to meetings of councillors and officers at over 100 local authorities over the years, and I’ve watched the body language: When you suggest that it would be a good idea for councillors to use easy-to-use web-tools, the more senior officers in the room start to futz with their biros.

There are all kinds of ‘budget maximising’ bureaucratic reasons for this resistance, and I’m sure that this is partly Mr Cameron’s target here. But I’d suggest that there is a bigger obstacle here.

Demands for ‘transparency’ from public-sector bodies are fundamentally unfair for the reasons I’ve set out here. They are one-sided. Perhaps the biggest underlying theme in modern politics is the question of who gets taxpayers money? Is it the public sector, or their competitors in the private sector?

It’s a profoundly political decision. To demand transparency from one, not the other is to take sides, and I suspect that this is what the Conservatives are doing here.

Given the restrictions of commercial confidentiality, we can’t easily demand transparency from the private sector (though reformers demanding less short-termism in investment decisions may disagree).

One thing we can demand – from all economic actors – is more interactivity. If interactivity, not transparency were the war-cry, it would be a fairer, clearer instruction to give to everyone.

Peter Levine says…

“Although I acknowledge the value of expertise, we can identify several important general reasons why it is never enough and we always need citizens’ participation to tackle social problems.”

What follows is a list of three reasons why experts shouldn’t be allowed to make decisions on their own. It’s one of the best posts I’ve read on the subject, and rather than spoil it, I’ll urge you to read it all. However, there’s one issue that I’d have near the top of any such list that is missing (to be fair, the post is called Part One, so maybe Part Two is devoted to the question I’m about to raise). It’s this:

• Experts explain awkward issues satisfactorily to other experts. Their explanations are less useful to the lay-person
• Sometimes the public are given a glimpse of the experts’ explanation. More often, they see it through the dysfunctional prism of newspaper journalism
• Politicians – usually generalists -  then have to field an unsatisfactory briefing in the context of a bloody awful report in a newspaper – one that has been seized upon and further distorted by a pressure group of some kind.
• The politician then has to take the consequences of not taking the decision that the newspaper / pressure group prefers. If s/he does this successfully, they may only be substituting a very bad policy with a quite bad one (i.e. one based upon a partial understanding of the expert’s brief)
• And whatever happens, s/he has to bear any consequences of the policy’s failure

However, if more people are able to get at the expert’s advice, mash it around into something that proves to be a more accessible explanation (something that enables to politician to understand what the expert was really saying), then a more participative polity has improved an outcome. It can help to break the hold that newspapers and pressure groups have in describing problems, and this can only be a good thing, surely?

As an aside, Chris Dillow is often very good at dismissing experts in his own inimitable fashion.

Update: Before posting this, I saw that Peter has published part two of his critique of expertise. And he’s interviewed here on blogtalk radio.