You’d have had to be off planet not to have heard about this political disaster that blew up yesterday which led to Chancellor of the Exchequer Alistair Darling apologizing in the House of Commons.

I watched the whole thing live on BBC Parliament TV on my PC, from which the above screenshot comes. And I twittered my reactions in real time over the course of an hour (transcript below the fold).

Quick summary: Two data disks containing confidential financial and personal information on 25 million citizens of the UK – nearly half the entire population – have gone missing. Lost in transit, it appears, in between a government department in north-east England and its destination elsewhere in the country.

The BBC has good summary information on this sorry affair with links to more detailed coverage if you want it. And with so many media worldwide covering this story, you can take your pick from all the many gory details.

The latest development today is that Prime Minister Gordon Brown made his own apology to the House this afternoon and announced checks on every government department on how they handle individuals’ data.

Clearly there will be severe political consequences from this massive cock-up. One senior government official – Paul Gray, chairman of Revenue & Customs, the government department at the heart of this mess – has already fallen on his sword.

Incidentally, the link above for Paul Gray is to his bio in the page on the Revenue & Customs’ website listing the board of directors. Gray is still listed as I write this, two days after he quit.

So what to make of all this?

On the face of it, it beggars belief that information about individuals, their personal information including addresses and bank account details, and records of their claims for child benefit has been treated in such a cavalier manner.

It’s not just the almost unbelievable act of simply sending off a couple of CDs via a commercial delivery service.

It goes far deeper than that as it seems quite clear that Revenue & Customs have been cavalier for a very long time with regard to their treatment of confidential data about people.

Among much of the politically-focused commentary on this affair – and if you want pure political commentary and opinion, all of it pretty subjective, there are any number of places you can visit – I see a consistent view threading through all that commentary.

And that is – the civil servants who run government departments like Revenue & Customs have no real idea on how crucially important it is to safeguard information about people where such data has significant value and the risks of misappropriation and misuse carry enormous consequences for individuals.

So if anything positive comes out of this disaster, it is a loud and clear wake-up call to get houses in order (and clean those houses as required) so that the least thing you’d need to be concerned about is how the government looks after your information.

In the case of Revenue & Customs, that department may need a great deal of work in changing a culture of ignorance (as I see it) if anyone is to have any trust or confidence in them in the future.

I don’t think they can do it by themselves and not quickly, so it will become a matter for the regulators – especially the Information Commissioner – to force change, and rapidly, but focused.

In a strong commentary railing against the government over this affair, The Times has a good point:

[…] It is therefore essential for regulators to concentrate their attention on the big issues that are really important — protection against wholesale data losses of the kind at HMRC, protecting the taxpayer against enormous losses of the kind now threatened by Northern Rock — while leaving individuals and organisations as free as possible to get on with their lives.

Ah, yes, Northern Rock. That’s another big problem for Alistair Darling.

Watching Mr Darling in the House of Commons yesterday afternoon, I thought he handled the immediate stage of the unfolding lost disks crisis quite well, as I noted in my twittering.

Yet I do think he is hanging on to his job (and perhaps his political career) by his fingernails. He is where the buck stops as far as Revenue & Customs is concerned; if any further revelations emerge about that department, I’d say he’s had it.

And if the checks on all other government departments turn up other alarming states of affairs regarding data protection, Prime Minster Gordon Brown’s future may not be too rosy.

Rightly or wrongly, people will blame ‘The Government,’ not individuals in departments.

It could be that we may have a general election a lot sooner than we otherwise thought.

Twitterings, p.m. November 20, 2007:

• Watching Parliament TV live. House of Commons filling up, Alistair Darling started making statement re missing 15m ID disks. 03:32 PM
• He seems to be laying the ground for blaming ‘junior officials’ for not following procedures. 03:33 PM
• Says package being transported by TNT didn’t reach its destination. 03:34 PM
• Big internal investigation past week not turned up the missing disks. Now the police called in. 03:36 PM
• 25 million individuals’ data lost, not 15m. Includes bank info. Wow. Serious stuff. 03:37 PM
• Now talking about other data losses by Revenue & Customs in September. 03:42 PM
• Apologizes for ‘anxiety that will undoubtedly be caused’ to millions throughout UK. 03:44 PM
• Conservatives on the attack now… 03:44 PM
• Pretty wimpy response by the Conservatives’ Osborne. 03:48 PM
• But I like Osborne’s final comment to Darling: ‘Get a grip.’ 03:49 PM
• Roars of laughter from MPs when Darling said info in planned ID cards would be safeguarded. 03:53 PM
• LibDem’s Cable asking how many unencrypted CDs do the govt send around departments. 03:54 PM
• Cable asking why govt IT contractors get better data protection than 7.5m families. 03:55 PM
• Darling: this data should not have been transported the way it was. Well that’s pretty clear! 03:57 PM
• ‘Individuals concerned ignored instructions.’ Yep, looks like the path for blame is being laid. Which is not where the buck stops. 03:58 PM
• Darling talking about need in future to go where data is not transport data to you. 04:00 PM
• Edward Leigh: ‘Chancellor may have inadvertently misled the House’ re the dates when data disks tranported. Says not fault of NAO. 04:02 PM
• The missing data is on 2 disks, says Darling. 04:03 PM
• Good grief! Replacement disks sent, this time by post. Unbelievable. 04:04 PM
• Discussion still going on in the Commons. Into lots of detail now. My take 1: Darling did a good job overall in explaining what happened. 04:06 PM
• Take 2: something seriously, but seriously, wrong at Revenue & Customs in how they safeguard data. 04:07 PM
• Take 3: if you do online banking, expect slow access times on bank websites for next day as everyone goes online to check their info. 04:08 PM
• Looking at faces in the Commons. Everyone glum-looking, unsurprisingly. And no one making big political capital. Well, not yet. 04:12 PM
• Conservative spokespeople keep bringing up ID card plans. Ah, political capital point-making. 04:15 PM
• Blimey! Looks like the databases themselves are not encrypted, never mind a couple of disks. Who the hell runs IT at these govt departments? 04:17 PM
• Ian Duncan Smith: if data does get into wrong hands, will Chancellor and Prime Minister resign? 04:18 PM
• Darling spoke a lot in reply but didn’t answer the question. Well of course not :) 04:19 PM
• One MP saying that code breaking of data is easy, look at what happened with the iPhone. Some MPs up to speed with latest tech :) 04:20 PM
• Ok, back to work while Commons Q&A goes on. Recommend Nick Robinson’s blog for when he has commentary on all: http://tinyurl.com/q4cam 04:25 PM