countculture

Yesterday I was invited to a meeting at the Department for Communities and Local Government with the key players in the local spending/Spikes Cavell issue that I’ve written about previous (see The open data that isn’t and Update on the local spending data scandal… the empire strikes back).

The meeting included Luke Spikes from Spikes Cavell as well as Andrew Larner from IESE (the Regional Improvement and Efficiency Partnership for the South East), which helped set up the deal, as well as myself and Nigel Shadbolt, who chairs the Local Public Data Panel and sits on the government’s Transparency Board. I won’t go into all the details, but the meeting was cordial and constructive, produced a lot of information about how the deal works and also potentially made progress in terms of solving some of the key issues.

We can now, for example, start to understand the deal – it’s called the Transform project – which as I understand it is a package deal to take raw information from the councils accounts and other systems (e.g. purchase & procurement systems) to SC’s specification, clean up and depersonalise the data, then analyse to show the councils potential savings/improvements, and finally to publish a cut of this information on the Spotlight on Spend website. Essentially we have this:

There are still some details missing from this picture – we haven’t yet seen the Memorandum of Understanding which frames the deal, nor the specification of the raw information that is provided to Spikes Cavell, but we have been promised both of these imminently. This last one in particular will be very useful as it will allow us to refine the advice we are giving councils about the data they should be publishing in order to make the spending information useful and comparable (it’s not been suggested previously, for example, that it would be useful to include details from the council’s procurement systems, though in hindsight this makes a lot of sense).

Crucially, it was also agreed that all the input data into Spikes Cavell’s proprietary systems (the ‘Cleaned-up but non-proprietary data’ in the diagram above) would be published, so the wider community would be on the same footing as Spikes Cavell as far as access to the raw data goes. This is crucial and worth repeating: it means that anyone else will have access to the same base data as Spikes Cavell, and the playing field is therefore pretty much level.

There are still issues to be sorted out, the chief of which is that while Spikes Cavell is happy to publish the raw data under a completely open licence, they will require the OK of the council to do so. (However, armed with this knowledge it will be easy to identify those councils that refuse, and then possible to tackle them either through persuasion or ultimately legislation.)

The other issues are, briefly: liability for depersonalising the data; where the data is published (I think it should be on the council’s own website or a data.gov.uk, or for London councils the London Datastore, not on the Spotlight On Spend website); whether the Spotlight On Spend website itself is necessary and cost-effective (it’s impossible to know how much it costs as it’s bundled in with the whole deal); and whether the data-cleansing should be stripped out from the rest of the deal.

However, it’s worth saying that this agreement goes beyond just the member councils of the IESE, but to all councils that in the future use a similar agreement (obviously it’s ultimately up to them, but certainly this was the wish of everyone at the meeting).

Finally, I’d like to thank Andrew Larner at IESE for his open approach, and for Spikes Cavell for their willingness to engage. What we have here isn’t perfect (and I still fundamentally believe that councils should be doing the cleansing and publishing of the data themselves, and exchanging that knowledge with other councils and using it to improve their own data processes), but it’s a big step forward in genuinely opening up raw council data.

Update: The official notes of the meeting have now been published on the Local Public Data panel blog: http://data.gov.uk/blog/local-public-data-panel-%E2%80%93-sub-group-meeting-spotlight-spend-20-july-2010