Law, code and architecture

Law matters.

It matters at a detailed level because we all need to understand rights, duties, obligations and constraints. It matters at a social level because it is the framework for mutually accepted constraint which is part of what defines civilised society.

The purpose and foundations of law are lofty and quite abstract. The immediate practicality of law is often hugely detailed, complicated and largely incomprehensible, even to specialists. That’s not a new problem – as Edward VI put it almost 500 years ago:

I wish that the superfluous and tedious statutes were brought into one sum together, and made more plain and short.

That line is taken from – and this post prompted by – the launch of a Cabinet Office initiative on Good Law which aims to ensure that law is necessary, clear, coherent, effective and accessible. It’s hard to argue with those characteristics. But anyone who has ever needed to understand legislation will know just how far the statute book is from representing them.

This post is a reflection on that event. It puts forward an idea for making law more usable which I am fairly sure is absurd and unworkable, in the hope that at the very least it might give those whose job is to think about this problem a fresh way of framing the question – and if by some miracle there might be more to it than that, so much the better.

Update: the video of the event has now gone up at YouTube. The whole thing is two hours, including questions and discussion, but well worth watching at least the opening presentations. There is also a lively discussion among the contributors to a live chat at the Guardian public leaders network.

Some law is simple and powerful. Years ago, when I worked on benefit fraud, the Theft Act 1968 was one of the tools of our trade. Its opening words distill its essence:

A person is guilty of theft if he dishonestly appropriates property belonging to another with the intention of permanently depriving the other of it

Some law is far from simple. It is hardly possible to sustain the basic clarity of that definition of theft in creating the 4,000 further criminal offences which were apparently enacted between 1983 and 2009. The loss of simplicity may result from the detail and complexity of what it is trying to address, for example the obligation for a mariner to pay national insurance contributions may depend on whether his ship has sailed beyond the River Elbe. But all too often the complexity comes from the structure of the law itself. A power may be inserted in one act by a provision of a second act to produce regulations which may have effect in part by amending a third act or any number of other sets of regulations. It is possible to navigate along those chains, but it is not easy. The diagram below, taken from the good law report shows the web of effects of just one Act, the Companies, Audit, Investigations and Community Enterprise Act 2004. It is not a pretty sight. Diagram showing complex network of connections between different pieces of legislation So the fact that there is a problem is clear enough. The question is what can be done about it. I will not attempt – and am not qualified to attempt – anything like a comprehensive answer to that, but I do want to reflect on one aspect of the problem.

Legislation does not spring fully formed from the heads of parliamentary counsel (and secondary legislation does not spring from there at all, but let’s keep things simple). It is the culmination of long and complex processes which express the underlying intention in different ways at different stages. The general direction is usually from the more general to the more specific – green papers go white, primary legislation generates secondary.

Legislation is far from being the only area of life where this is true. The development of any system or process will tend to go through some version of it, it just won’t have legal force at the end. One of the more obvious parallels is with software development. Commonly that’s seen in terms of law as a logical system, typified by this remote contribution to the good law launch:

I am a former computer systems analyst and now a solicitor. I believe that reducing statutes and case law to a series of nested if-then propositions is possible and would make the law far more accessible to anyone who is able to access a computer. Such computer systems used to be called expert systems. With this new initiative has their time now arrived?

There is a lot of power in this view. In principle such an approach should support a much more direct link from at least some kinds of law to implementation. There are powerful tools based based on just that perception, such as Oracle Policy Automation, which are already starting to be used in some parts of government.

But in this context I am less interested in law as code and more in law as coding. To whatever extent law is like software, is drafting law like writing code?

As John Sheridan pointed out at the launch event, the process of software development has become increasingly sophisticated, not just in the availability of tools, but also in softer techniques such as pair programming. Could similar techniques help create clearer and better structured law?

One starting point is to consider where the definitive version of any piece of software is to be found. Ultimately there is some machine level code which is the set of instructions actually followed by a computer. It is though incomprehensible to humans. Sitting above that is source code which is written by and comprehensible to humans – but only highly expert ones (and the more powerful the software the less straightforwardly comprehensible the source code will be). A step back again, there may be documents which capture the logic of the intended system. Critically in this context, that is the first level for which the primary audience is human. Beyond that, there may be various documents capturing requirements and intentions, but those are descriptions of the intended system, not representations of it. For a system of any complexity, there will also be some kind of architecture which sets out the role of each component and how each relates to all the others.

There is no arguing with machine code: it does whatever it does, and so is undoubtedly definitive in one sense. But in a different way, there is no arguing with higher level requirements: they should be the definitive description of what is intended.

If you want to know how a system works and what it does, you are likely to be much better off starting with architecture and working down towards code than you would be by starting with some arbitrary code and trying to infer architecture.

How does any of that map to legislation? Source code is probably the closest match to legislative text. General requirements are, perhaps more loosely, analogous to policy documents. But what of more detailed specifications? And where is there any sense of architecture?

As it happens, there is a place in the legislative process where high level system logic is captured, though one I don’t think was mentioned at the good law event – instructions to counsel. These are curious documents, with no constitutional or legal status but immense practical significance. They are supposed to be the most complete, rigorous and systematic specification of what the legislation is intended to achieve and in theory (though I suspect vanishingly rarely in practice) should be all that is needed to allow parliamentary counsel to draft legislation which correctly implements ministers’ intentions. In order to do that, they need to operate at several different levels at once. They specify the system logic – the if-then statements both for the policy core and for the inevitable edge conditions. In addition, they both communicate the intention, against which more detailed parts of the solution can be tested, and also set that in context, and may illustrate what is intended in one area by analogy with what has previously been done in another.

Nobody would pretend that they are a light read, but they do have real advantages over the legislation which they generate. The first is that they are comprehensive: the intention is in a single place, even if the legislative implementation is scattered over new and amended statutes. Secondly, they are continuous prose: they tell a story in a way which the more code-like text of legislation will never equal. Thirdly they are more easily testable: it requires less specialist knowledge and expertise for a minister or policy official to ask themselves whether what the instructions describe is the outcome they intended to achieve. And finally, they are rigorous: much of the virtue in producing instructions lies not in the finished product but in the sometimes painful process of being made to translate implicit or unspoken aspects of the policy into explicit requirements.

A good set of instructions, therefore, is a robust description of the intended legislative effect. The actual draft legislation is either a faithful representation of the instructions, in which case it can have no greater information content than the instructions did to start with, or it fails to be a faithful representation, in which case it is wrong (I am, of course, massively over-simplifying here – ignoring both how legislation changes through its parliamentary passage and the fact that there are already established – if exceptional – ways of ascribing meaning to law other than just through its own language). So we can ask again, which is the definitive version. In the world we know there is only one answer: it is always the legislative text. But perhaps there are advantages in creating a world where the answer is not so simple.

What if we were to give primacy not to the legislation but to the instructions? Put like that it sounds absurd, but what if we were to ask instead, what if we were to give primacy not to the technical coding, but to the agreed and understood system requirements.

The effect in one way would be related to purpose clauses and recitals, both of which try to make clear what the legislation is intended to do before getting into all the detail of doing it – though neither of which seemed to find much favour in the discussion. But it would be a much bigger change than that. Perhaps the most powerful feature would be that they could start at the level of architecture, embodying a version of the network diagram at the top of this post. They would map the landscape to set the reader’s bearings before diving down into the greater detail. They would shift the political debate, with the primary question always being whether the intention is clear and has been fully captured, rather than time and energy being spent on technical amendments which nobody understands.

More subtly, they offer some chance of bringing some of the narrative clarity of case-based law to the more arcane structures of statute-focused law. Richard Heaton made a powerful point that much law is taught and learned through stories, the cases which bring the law to life and set its boundaries in a way very different from the endless technical detail of administrative and other areas of law. That immediately makes it easier to engage the interest and contributions of the great majority of people who are constantly affected by laws of which they have little understanding and over which they have little influence.

So my modest proposal is this. Recognise instructions to counsel for what they are: the definitive statement of what the law should be. Let that be debated by parliament – and by all the rest of us. Treat everything we currently see as legislation as technical implementation of that architecture, still vitally important, still needing the highest quality of professional skill in its construction and application, but of no more interest to non-specialists than the source code of a browser is to those reading these words.

This may be a completely ludicrous idea. I am more than half inclined to think it is myself. But let me end with a thought experiment. If we had the freedom to start again completely, without any sense of how law should best be made and without any burden of history or tradition, what might we then invent? And would that be more like the Westminster model we are all so familiar with, or might it be closer to the approach I have sketched out here?

Source code picture by Sebastian Delmont, licensed under creative commons

A short history of computing, by a boiled frog

Posted by Public Strategist

Today my home broadband got upgraded. A few hours ago, 7Mbps down and a tenth of that up was as good as it ever got. Now it’s close to 50Mbps down and, even more dramatically, about 15Mbps up.

Once there were no computers visibly in my life, still less devices which happened to compute. Now they are ubiquitous and their impact has been transformational. The whole change has been enormous, but the scale of it is rarely obvious. No one step was life changing, but tracing back through the journey gets us to a strange and distant place.

I remember…

I remember when I first felt that I had the whole internet with me, almost always and almost everywhere

and stopped reading a newspaper on the bus to work every morning, breaking one more connection with physical information

I remember when I first got a smartphone

and left for a long trip in which I was able to manage flights, hotels and maps with a device which slipped into my pocket – and could even make phone calls

I remember when I first got a tablet

no, not one of those, one of these

I remember when I first started on twitter

when I realised that everybody at a meeting I was at had been having – and were still having – an interesting conversation which I wasn’t part of and couldn’t see was happening

I remember when I made all my CDs vanish

but could listen to them all over the house, using what to this day are some of my favourite ever gadgets

I remember when I first got broadband

it was partly the speed (though that was a tiny fraction of even yesterday’s standard), but even more so the immediacy of the connection: you didn’t have to go online any more, you could be online, and that made all the difference (and not paying by the second helped too)

I remember when I first got internet access at work

on a standalone computer in a locked room at the the end of a corridor, with a book in which to write a complete list of sites visited

I remember when I first got ISDN

you could have two 64k channels for a blistering 128k – but at the cost of two phone calls, so every second had to count

I remember when I first took a digital photograph

on a camera made by a company called Kodak, which used to be in the photography business

I remember when I first got a computer at work

which required a business case for each PC individually, constrained by the fact that typists were to type any document over a hundred words

I remember when I first accessed the web

it seemed enormous, but Yahoo was still trying to keep a central index of websites up to date by hand

I remember when I first went online

in what would now be called a walled garden, but for a while the inside was almost bigger than the outside, and it was suddenly possible to get something out of a computer I hadn’t put in to it

I remember when I first saw a mobile phone

a stranger in a pub, whose phone took up an entire attaché case, and who spent ten minutes setting it up and turning it on, but didn’t seem to have anyone to talk to

I remember when I got my first computer at home

with two floppy disk drives (now down to only 5¼”)- adding the optional 10Mb hard disk would have made it unaffordably more expensive

I remember when I first used a word processor

amazing dedicated machines, superbly optimised for their single task, using 8″ discs to store a few dozen pages of text

I remember when I saw my first home computer

built by a friend from a kit, playing jerky space invaders from a cassette tape

I remember when I first used a computer terminal

a teletype from school over a fixed line to the local technical college, programming first BASIC then FORTRAN on paper tape, almost always in batch mode with 24 hour turnround, so typos really mattered

I remember when I first had a pocket calculator

which had the extra sophistication of a percentage key as well as the four basic operators, and was called a vatman as a result

I remember when I had a book of four figure log tables.

None of that was very long ago. Or so it seems to me.

The black screen of life

Posted by Public Strategist

Every now and then I still get a slight frisson from the thought that I can get things out of my computer that I didn’t put there.

My first computer was very simple. You turned it on and got to a screen which said, in its entirety:

A:\>

(note to younger, but not very young, readers: no it didn’t say ‘C:\>’, that was more than I could afford)

One of the two key pieces of software I used back then was even more terse. It’s opening screen was:

.

(note to readers with dust or dead pixels on their screens: that’s a single dot, not a complete blank)

After that, everything which came out was a product of what had gone in. Documents came out after they had been laboriously typed. Data analysis came out (and in short order an entire work management system, but that’s another story) after not only had the data been keyed in, but all the rules and structures as well.

Time passed. That computer got replaced. A modem was acquired. And suddenly, stuff came through my computer that I hadn’t put there. That was extraordinary, exciting, and more than a little magical.

Much more time has passed. Computers have been replaced and replaced again. The warbling modem became a silent, speedy, but expensive ISDN connection, then broadband arrived (and though the speed has since changed, that first ADSL router is still chugging away after more than ten years of constant service).

And thresholds of what is extraordinary, exciting and magical have gone up a few notches:

Online check-in still seems rather magical to me.

26 October 2010 10:40 am via TweetDeck ReplyRetweetFavorite

@kcorrick

Kathryn Corrick

Technology, it is said, is everything that was invented after you were born. I have always liked that idea, but perhaps it is no longer adequate to capture the rate of change, which is why Douglas Adams’ variant is even more seductive. And in the meantime, plain old online check in is starting to feel a bit quaint - if your boarding pass isn’t on your mobile, it’s not magical at all.

What are we creating today which will still feel magical tomorrow?

Barriers to progress

Posted by Public Strategist

This is not a post about the barrier I came across as I walked to the post office on Saturday.

I had to turn into the path in the picture, which takes a 210° hairpin to the right, followed by a 110° turn back to the left – it’s sharper and more awkward than the picture makes it look. A moment’s trivial inconvenience, a moment of minimal exasperation, and on I went – as no doubt does everybody else who needs to go that way.

I am going to assume – perhaps generously – that whoever decided to put it there did not do so with the express intention of creating an unnecessary obstacle for pedestrians. But if not that, then why?

The obvious explanation is that it is there to discourage cyclists from using the path as a short cut, and for all I know, it succeeds at that perfectly. But that’s not an answer to the much more interesting question of whether the existence of the barrier is a good thing.

I think there are at least three factors which need to be looked at here (there may be more, but that’s plenty to start with):

Effectiveness
Externalities
Harm avoided

Effectiveness is the easy one. If it is designed to discourage cyclists, does it in fact discourage cyclists? The fact of the obstacle is indisputable. It’s not impassable, because there is a necessary trade off with pushchairs and wheelchair users, but it’s probably enough to be mildly discouraging.

Externalities are reasonably clear too. The fact that I am bothering to go on about this at all is an indication that there are some. Even I couldn’t argue that they are big for any single pedestrian. Whether they are cumulatively bigger than the cumulative benefit of discouraging cyclists is another matter. Since whatever cyclists there might have been have been discouraged, it’s hard to tell.

That gets us to harm avoided. How bad would it really be if the cyclists were not discouraged? I have a feeling that the answer may well be, “not very”. That’s partly because similar barriers which get in the way of where cyclists actually want to go seem to have no effect whatsoever (I say that with no malice – this is not a post about, let alone against, cyclists – merely as an observation), and partly because in only a few yards (only to the end of the railings on the left) of shared use of a wide path, the scope for conflict should be limited.

Since only a small proportion of the readers of this blog can be supposed to have an obsessive interest in the minor footpaths of south London, that’s probably more than enough on that. The real question, of course, is what this might tell us about some bigger and less obvious problems.

I think it’s the last of the three factors which most often trips up decision making. If the problem is assumed to be big, the solution will tend to be big too. And even if the problem is big, it is not unusal for it to have been made big by something which is treated separately. So, to take another local environment example, the road where I live has speed humps to slow the traffic down. But it only needs slowing down because the installation of traffic lights sped it up.

So step one should be, is the problem as big as it looks?

Step two then becomes, can we make the problem smaller?

That leaves as step three, is the problem now small enough that we can accept the risks it carries because of the positive externalities?

Reflecting further on my own recent post reinforced by some thoughts from Lost ConsCIOusness, I wonder how far skewed outcomes are a result of looking too much at effectiveness and too little at harm avoided and externalities. If the size of the problem is exaggerated, purported solutions to it will be disproportionate. And if the costs of the solution fall on users of the service or on random passers by, there is no incentive to reduce them.

On the slow train to the future

Posted by Public Strategist

I am on a train, going to a meeting about using the next generation of technology in the workplace to improve the effectiveness with which we do business. Next generation in this case means the generation after Windows XP and Internet Explorer 6, so I am containing my excitement, but there is no doubt that there are opportunities to do things better and to do better things.

I am on a train, typing on a device which does a perfectly reasonable job of looking like a laptop while doing the absolute minimum possible to behave like one.

Back home, I have massively more useful technology which is set up to do the things I want to do in the way I want to do them. I could have brought my own laptop with me, but my bag is heavy enough already, and the office laptop is the only way of getting to office (and indeed Office) email. So this post from Bruce Schneier speaks to me powerfully:

If you’re a typical wired American, you’ve got a bunch of tech tools you like and a bunch more you covet. You have a cell phone that can easily text. You’ve got a laptop configured just the way you want it. Maybe you have a Kindle for reading, or an iPad. And when the next new thing comes along, some of you will line up on the first day it’s available.

So why can’t work keep up? Why are you forced to use an unfamiliar, and sometimes outdated, operating system? Why do you need a second laptop, maybe an older and clunkier one? Why do you need a second cell phone with a new interface, or a BlackBerry, when your phone already does e-mail? Or a second BlackBerry tied to corporate e-mail? Why can’t you use the cool stuff you already have?

More and more companies are letting you. They’re giving you an allowance and allowing you to buy whatever laptop you want, and to connect into the corporate network with whatever device you choose. They’re allowing you to use whatever cell phone you have, whatever portable e-mail device you have, whatever you personally need to get your job done. And the security office is freaking.

Mine is an organisation with more than the ordinary tendency to – and justification for – freaking. But the security model, and the wider organisational model from which it derives, are looking increasingly brittle. We can no longer afford to pay large sums for greater disfunctionality. Something will have to give.

That’s as far as I got on the way this morning. Now I am on the way back. Not much has changed, except that it’s late enough for it to be not worth the hassle of battling the woeful combination of appalling mobile service along this line and a laptop setup which means starting from scratch every time the connection coughs. Unread emails will stay unread for a while longer.

The real question, of course, is not whether I should be allowed to create my working environment and link it with the department’s systems. I am pretty clear that I should – but equally sure that that puts me in a pretty small minority (but in five years? ten?). Big organisations tend not to be good at catering to small niche requirements, so that wait will continue. But that does not mean that the subversive impact of what ostensibly started as a routine and unavoidable technology update will not be powerful and ineluctable. The introduction of new tools always gives more power to those best able to use them – and they are rarely those who were the masters of the previous toolkit. That much is standard innovators’ dilemma territory. There is any number of wider effects though, of which three are particularly in my mind at the moment:

Trust and self-control When I wrote about the socially mediated workspace a couple of years ago, it was to make the point that it was no longer just the case that home technology is often more modern and more powerful than office technology, but that the disparity is increasingly about the social use of technology, rather than the technology itself. But the power of social tools to amplify knowledge and connections is only available if organisations and managers are much less controlling, and individuals accept that that makes responsibility squarely rest with each of them – though see that earlier post for some thoughts on why that may be harder to do in some organisations than others. Trust must first be given and then earned. It doesn’t work the other way round.

Knowledge is not power For bureaucracies in particular, knowledge has often been power. Being a gateway in a traditional organisation is not at all the same as being a connector. That will never entirely change: not everybody can know everything, and some forms of knowledge and of the skills to make effective use of it will always be personal and valuable. But knowledge about knowledge is a different matter. What gatekeepers do in practice is often to control knowledge about where the knowledge is, as much as they control the knowledge itself. If everybody has effective tools for finding what they need, the consequences are enormous.

The web is the natural unit of organisation I don’t mean by that that the browser should be the gateway to everything, but that many to many connections are ever more important. That is very different from the hegemony of the organisation chart. Distributed knowledge will reassemble and reconnect itself in unpredictable and powerful ways.

None of those three is a necessary consequence of updating technology. We can probably cling on to the twentieth century for a while longer if we really put our minds to it. Nor are they the only or the most important consequences – I don’t pretend to be able to predict what those might turn out to be. But they are to me a sufficient consequence: sufficient to make it more than worthwhile to invest effort not in making them happen (I am not sure that that can be done), but in tackling things which get in the way of their happening.

This could all get quite exciting. All we need now is the shiny new kit – and only another year to wait.

The end of history

Posted by Public Strategist

The end of history will come not when nothing more ever changes, but when nobody can work out what has actually happened. We may be closer to that point than we like to think.

In the world of organisations, historically, the creation of records was a by-product of actually doing the work. It wasn’t hard to remember to do the filing, because the file – and before that the ledger – was the unit of activity. Sorting out the files of a few years before to weed out the ephemeral was a bit more of a chore, but no more than that. And while there might be newer and shinier filing cabinets, and while metal was replaced by plastic at the ends of the treasury tags, fundamentally nothing had changed for decades. Historians of today are still in that world since they operate with the time lag of the thirty year rule, and it will be a few years yet before the reduction to twenty years makes much difference to that.

James Lapping has recently argued that in each of the last eight centuries there has been a standard way of keeping records, but that the explosion of systems and technologies around the beginning of this century means that those standards have vanished:

In the 20th century there was competition between companies supplying organisations with paper, filing stationery and filing cabinets. But none of those companies were trying to disrupt the way individuals communicated or stored information. No company was trying to come up with an alternative to the hard copy document, or an alternative to the practice of gathering hard copy documents together into files. They could not have been able to disrupt these practices even if they had wanted to.

In the 21st century there has been a constant stream of new formats and applications disrupting the way people record and communicate information. Documents are still important. But successive waves of new formats (e-mail, instant message, blogs, discussion boards, wikis, status updates) have come in alongside the document.

I doubt that there has ever been a time when people created records for the sake of it beyond any immediate purpose they may have had for them (though I say that with the insight of a practising bureaucrat, rather than with the knowledge of a historian). If that’s right, it’s probably a safe bet that our successors will be no more motivated to do so than we are. So if progressing the work continues to diverge from creating records of what has been done, the raw material of history may be thinner in future than it has been for centuries (and history here means medium term institutional memory as much as it does the work of historians). That problem will not be solved by exhortations to do better filing: it will be solved, if at all, by tools which support what people are trying to do in the short term while quietly adding what may be needed for the longer term – which is easier said than done, as James Lapping notes:

If you can’t predict what format or what applications people are going to use within your organisation then your only option is to have a system that will manage records produced in any format, in any application. [...] However there are a great many unanswered questions about the model. Even if the functionality is there, no-one knows how organisations could make it work. There are no case studies out there that I have heard of, and no guidance notes from any national archive or professional society.

On the face of it, all that is a strange state of affairs. In the wider online world, the worry is that we will remember too much, not too little, with ever greater concern being expressed. As the Wall Street Journal recently reported an interview with Eric Schmidt, the CEO of Google:

“I don’t believe society understands what happens when everything is available, knowable and recorded by everyone all the time,” he says. He predicts, apparently seriously, that every young person one day will be entitled automatically to change his or her name on reaching adulthood in order to disown youthful hijinks stored on their friends’ social media sites.

Even more pertinently, the New York Times ran a long essay a few weeks ago called The Web Means the End of Forgetting:

All around the world, political leaders, scholars and citizens are searching for responses to the challenge of preserving control of our identities in a digital world that never forgets. Are the most promising solutions going to be technological? Legislative? Judicial? Ethical? A result of shifting social norms and cultural expectations? Or some mix of the above? Alex Türk, the French data-protection commissioner, has called for a “constitutional right to oblivion” that would allow citizens to maintain a greater degree of anonymity online and in public places. In Argentina, the writers Alejandro Tortolini and Enrique Quagliano have started a campaign to “reinvent forgetting on the Internet,” exploring a range of political and technological ways of making data disappear.

Of course, that may be plain wrong, as Scott Rosenberg has trenchantly argued in a post on the NYT article

There is an entropic quality to everything that is shared online. Data gets lost; servers die; databases are corrupted; formats fall into disuse; storage media deteriorate; backups fail.The Web is now old enough for us to know just how badly links rot over time. Much of the material from the early days of the Web is already gone. Facebook and Twitter actually make it nearly impossible for you to find older material, even stuff that you’ve contributed yourself. The more dynamic the Web gets and the more stuff we move into “the cloud,” the less confident we can be that information that once was public will remain available to the public.

There are plenty of versions of this dilemma around once we start looking for them. Take the apparently simple case of how storing personal photographs differs between the twentieth century and the twenty first. Deane Barker reflected recently on being asked to select some photographs – why choose, he asks, when it is so easy to keep all of them. He concludes with the interesting thought:

So, image media has officially been reversed. The digital version used to be an annoying stepping stone to a physical version. Now, the physical version is almost a weird afterthought.

Almost simultaneously, and as far as I know with no connection, John Naughton was reporting and endorsing precisely the opposite argument:

Parr thinks that we should also print our pictures, and I suspect he’s right. “We are in danger”, he says,

of having a whole generation – and this will continue into the future – that has no family albums, because people just leave them on their computer, and then suddenly they will be deleted. You have to print them and put them in an album or a box, otherwise they could be lost. And write captions. You might think you are going to remember what is happening in a picture, but you probably won’t in 10 years’ time.

So where does all that leave our debt to posterity when, notoriously, posterity has done nothing for us?

The approach for the last ten years or so has been, in effect, to create an electronic version of the old paper filing system. The Modernising Government white paper stated in 1999 that:

It is our aim that by 2004 all newly created public records will be electronically stored and retrieved.

Some departments spent a lot of time and money on failing to reach that objective, others did little or nothing and also failed to meet it. What those systems demonstrated, in my very limited experience, was essentially that users were very intolerant of friction, of doing anything at all which got in the way of their immediate task. Confronted with systems which imposed the least degree of friction, they simply worked round the obstacle. I claim no expertise in this area, so it may be that the tools have now improved so dramatically that the friction has completely vanished, but whatever the degree of improvement, that still feels like a model locked in to an old way of defining the problem.

The old model assumes that archive capacity is a constrained resource. But what if it isn’t?

There are two aspects to that question: what we have the capacity to store and what we have the capability to find afterwards. Historically the two were closely linked: storage was physical and had to be highly organised in order to stand any chance of finding anything later. But on the face of it, both those constraints are now broken. We can store pretty much what we like, and even at eye-watering enterprise storage prices, I strongly suspect that it is frequently more expensive to go through the process of deciding not to store something than it is just to let it be stored (though it is even more certain that those costs fall on completely different budgets, that the second is visible and the first is not, and that as a result the trade off is not optimised).

As Scott Rosenberg put it in a piece I quoted from five years ago,

My in box is not a desk that must be cleared. It is a river from which I can always easily fish whatever needs my attention. Why try to push the river? Computer storage is cheaper than my time; archiving is easier than deleting.

Finding things again remains a challenge: Rosenberg’s argument about entropy and Lapping’s about the need to manage not just current formats but obsolete ones and those yet to be invented are both powerful ones. Even there though, the quality of search tools and the availability of the computing power needed to make them effective strongly supports the shift from the old approach to the new. It doesn’t matter how big the haystack is, if a search for ‘needle’ always returns the needle you are looking for.

History will, of course, look after itself. It always has. But the future history of our time will be different from our histories of past times, and that will not be because we have an eye to the future, but because we are always relentlessly focused on the present.

Image by curiousyellow, licensed under creative commons, some rights reserved.

Re: Re-Rewired State

Posted by Public Strategist

Rewired State has taken another step towards becoming the next generation systems integrator for government. In a piece of delightful recursion, a Rewired State project becomes the vehicle for accessing the formal status of Rewired State – or as it has been since last Monday, Rewired State Ltd.

In other news, the Rewired State gang has just announced their plans for March 2010. Last year’s National Hack the Government Day still sticks in my mind as something of a personal turning point. It was the first time I got a sense of the energy and opportunity represented by that community, and as I summed it up in my account of the day:

The simple fact that lots of smart people thought the best thing they could do with their Saturday was to think really hard about how to make government better is a force for good we cannot afford to lose.

This year, that’s just one of four events being run under the Rewired State banner. What’s really interesting is what appears to be an entirely non-accidental absence of any sense of groundhog day. The world has moved on in the last year – for which the Rewired State crew can take some of the credit. The question is no longer, can bright people do smart things with government data? That is proven beyond challenge. The question now is how those ideas can break through from demonstrator and prototype to robust and scalable service, and from services which are available and potentially useful to services which are used and celebrated. So it is really interesting to see that two of the four events involve paying developers to tackle specific problems while still leaving plenty of space for the more anarchic hack day itself.

I have got a longer post I had been writing over the last few days on some of these issues before any of this was announced, which should see the light of day imminently. But since I am absolutely unqualified actually to take part in Rewired State (the last time I did anything which looked at all like coding was in 1987, and it wasn’t many years before that that I learned error correction by hand punching paper tape…), it can’t be too soon to start blagging for a seat at the presentations.

Aphorism 15

Posted by Public Strategist

More on powerpoint… the more, in number and in detail, your bullets, the less I believe you know your stuff!

Valdis Krebs

When I was five, I was just alive

Posted by Public Strategist

Last week, this blog hit five years and 400 posts, just as it became apparent that blogs are history.

As this momentous milestone approached, there was a flurry of coverage of the latest Pew Internet Project report, on social media and young adults, picking up on the decline of interest in blogging – at least among young Americans. The Guardian reported that:

Blogging, on the other hand, may become more and more of a side issue. In fact, among all the content creating activities the decline in blogging among teens and young adults is striking as it looks like the youth may be exchanging “macro-blogging” for microblogging with status updates. Since 2006 blogging among teens has dropped from 28% to 14% and among young adults (aged 18 to 29) by 24% to 15%. Some 11% of those aged 30 and over now maintain a personal blog, and 14% of them maintain a personal website.

There is nothing terribly surprising about that: maintaining a blog is not a trivial undertaking, and it has always been true that a lot more people are consumers of online material than are producers of it. Kathryn Corrick recently picked up on some Forrester analysis (again based on the US) which shows this very clearly:

What has changed, perhaps, is that tools have become better tuned to what it turns out people actually want to do. As John Scalzi puts it:

For the vast majority of what people (not just teens, but teens also) used blogs for — quick updates on line to friends and family — Facebook and Twitter offer an easier, friendlier and therefore better solution than starting up a blog. If you’re starting out in social media, for most folks it makes sense to go there. Later, if you want the ability for customization and a format beyond 140-character tweets and status updates, you can always start a blog. But I suspect most people don’t need to get to that point, and certainly not most younger users of social media.

Also, you know. Blogs have been social media’s Last Year’s Model for a spell now; heck, they were Last Year’s Model when Friendster hit. And it’s certainly true that when I note that I’ve been blogging since 1998, certain younger folks get that look in their eye that says No! No one was even alive then! That’s when I hit them with the concept of “newsgroups.” Good times, good times.

Or, more pithily:

Great content is really, really hard to make. That’s why so few blogs have it, but that’s not the medium’s fault. The same is true for any other media.

And so back to the discussion of the state of the UK gov blogosphere kicked off by Dave Briggs, continued before and at UK Govcamp. Now Dave is back with some fresh thoughts (and with a great comment from Steph Gray), most importantly and perceptively that none of this is really about blogs:

I was wrong to mention blogs. A lot of the resultant discussion in the comments of that post and other chats have focused on blogging, which is of course just the medium. It’s the content I am interested in. What we seem to lack is an ecosystem of ideas in public services. Discussions about new ways of doing things, how to change the way things are, how ideas get progressed into prototypes and then into actual delivered services or ways of working. Whether this happens on a blog, in a social network, on a wiki or over a cup of tea is neither here nor there.

I think that’s a good way to approach the question, not least because the first incarnation of this blog was as the only available tool for the job I really wanted to do. Its original purpose was to act as an informal knowledge management exchange for me and my team at work. In the absence of any official way of doing that, a group blog – with access restricted to members of the group – seemed as good a way forward as any. For a whole range of reasons, it never quite took off in the way I had hoped, so it fairly quickly became more of a personal notebook of things I had found interesting or thought I might want to remember. That meant I wrote largely for myself – if anybody else found it interesting, that was a bonus, but their absence didn’t stop me (which is just as well).

Large organisations tend to be predominantly inward looking: there is so much going on and calling for attention on the inside that it can sometimes be hard to remember even that there is an outside, let alone that that is where challenge and innovation is most likely to be found (I read something interesting and thought provoking on that, using the pattern of email usage as the way in to the question, just in the last few days, but now I can’t find it to add a link here – which is itself a measure of one part of the problem). Blogs are one good way of countering that trend, for readers but perhaps particularly for writers, but its not the only way nor even the best way for many people and many ideas. As Steph says, there is an existing ecosystem (and set of assumptions) which long predates the world of social media. The challenge for government – and probably the challenge for any large-ish, non-technology focused organisation – is to recognise and embrace the additional power which comes from widening that ecosystem and, critically, to accept the loss of control which comes with it.

In the meantime, there’s a few more years blogging to be done.

Public Strategist

Working to make government work better

Category Archives: Work and tools

Law, code and architecture

A short history of computing, by a boiled frog

I remember…

The black screen of life

Barriers to progress

On the slow train to the future

The end of history

Re: Re-Rewired State

Aphorism 15