Open Data

David Megginson and Sean McGrath are both talking about open data. It's made me chuckle how little organisations value their data for a number of years. On many occasions it is saved. In whatever format it is originated in, proprietary, whatever. I've never met a manager who was concerned about the quality, freshness or accessibility (ability to get at it I mean) of that data. I'm absolutely convinced that if a suggestion had been made that time and effort be spent on the stored data I'd have been laughed at. I've met lots of engineers and others wanting to convert from Word processor format A to Word processor format B. Many of them annoyed that such work was necessary. Few seem to take that logical next step and look at the problem in the abstract. David lists a few of the common data transport needs. In terms of personal data that list could be extended. In terms of business data the same questions could (and should) be asked. What value is THAT data? Value to the organisation that is. What did it cost to generate? What is its lifespan? Is there a use-by date? For some organisations, producing information is their reason for being. Do they save their data? Do they archive it with a kill-by date? Is there a review date? Even more crudely is there any metadata at all! I've seen piles of paper, each meticulously identifiable, forming the build record of a product. I know this data is available on micro-fiche or print. Nothing else. I know of fifteen years of work, archived on disk, using an identification system that is little known and incomplete in, and of, itself.

It amazes me, or used to, that this happens. What I've yet to see is some sort of recognised professional waking around selling his or her services to organize that data. Do they exist? If they do, how would they help with the David Megginson problem list? Surely they'd just tell you what storage to buy and how to index the data? Who's interested in the data format standardisation?