Too much information

19 September 2011

In the continuing fight for the greater availability of public information, it may seem churlish to observe that sometimes what’s wanted is not more information, but less.

The picture above shows a typical display on a Countdown sign at a London bus stop. This particular stop has buses from two routes. At a quick glance, you might be forgiven for thinking that it’s going to be a long wait for a 2, while three 88s roll past ahead of it. Actually, the sign shows no such thing: the bottom three lines cycle through all the buses known to the system, so the second, third and fourth aren’t showing. Nor indeed are the eighth and ninth, which at that moment were 20 and 23 minutes away respectively. But if you want to know when the next 2 is due, you will have to wait  for the sign to cycle round.

I would be prepared to lay considerable odds that there is not a single person standing at that bus stop who has the slightest interest in the fact that a 2 will be along in 16 minutes, still less (if that were possible) that there will be another one in 20 minutes. What they care about is that the next one is three minutes away, and right now the sign isn’t telling them that. In extreme cases, there are so many buses stacked up in the queue that they come and go faster than the display can keep up with them – the screenshot on the right (from James Darling’s minimalist site) is from the bus stop with the most extreme case of that problem I have come across.

There are no imaginable circumstances when it would be useful to know that that the sixth bus to Marylebone is coming in 26 minutes, when you are already standing at the bus stop. But there is potentially much greater value in that information when you are not at the bus stop at all. So for Countdown at bus stops, perhaps the solution is simply to show less. The first and second bus due for each route served by the stop might do it (though capping by either time or an arbitrary number of buses shown wouldn’t).

But the moment you are not at a bus stop, the ideal solution changes radically, for two reasons. The first is that the information is now informing a different decision: whether now is a good time to be going to the bus stop in the first place, with opportunities for trade offs which exist only because the information has been liberated from the bus stop. The second is that the small screen in my pocket can comfortably show me the next twenty buses; the much bigger screen attached to the bus stop struggles as soon as there are more than four.

The point of this is one which at one level should be trivially self-evident: what counts as good information depends on the question you want to answer or the problem you want to solve, and more information may not be better information. But since there are many different questions and problems, it would be folly to think that there is a single best way of selecting and presenting the information. Part of the power of open data is that it creates the possibility for highly specialised solutions – as Adrian Short has both elegantly argued and practically demonstrated. James Darling’s account of why he built his version and what he and others have done with it is also well worth reading.

One of the comments to that post links to a more radical approach still: stop making it about the buses (tubes, trains…) at all, and turn it round to be about the passenger, in a form of extreme hyperlocalism:

As the about page says, if you live exactly 6 minutes from Sunset Tunnel East Portal, 8 minutes from Duboce and Church, and 10 minutes from Church Station you may find it useful too.

But enough. I have a bus to catch.

I had hesitated to write this post at all – there is only so much bus stop nerdery any self-respecting blog should contain, and I am already well over quota for this year.  But I took heart from Paul Annett’s meticulous deconstruction of a bus stop indicator at Heathrow, so here it is.


Aphorism 57

14 September 2011

When the data seem to point to an unexpected finding, always consider the possibility that the problem is a feature of the data, rather than a feature of the world …

When I discover something surprising in data, the most common explanation is that I made a mistake.

John Kay

Interesting elsewhere – 8 September 2011

8 September 2011

Things which caught my eye elsewhere on the web

  • My speech to the IAAC | Ben Hammersley’s Dangerous Precedent In the time of revolution, and believe me this is a revolution – easily on a par with the renaissance, or the Enlightenment – the translator has a very important role to play. The communicator, the person who makes the facts palatable to all sides, is the only conduit through which real change can be made.
  • Nick Bradbury: Privacy is Important Today’s software developers need to look at privacy the same way they’ve learned to look at security: it’s not an add-on or a feature that customers have to turn on, it’s something built-in that shouldn’t be turned off.
  • Seth’s Blog: The obligation of the adjustable displayThe Catch 22 of engineering feedback: “The only person smart enough to understand this warning doesn’t need it.” That’s over, I’m afraid. You have unlimited paper and a pen with plenty of ink. Be clear, enunciate and tell us what to do, please.
  • FixMyTransport emphasises flaws of government IT | Guardian Government Computing | Guardian Professional What makes FixMyTransport different to some of government’s sizeable IT failures is that it focuses on the user and what they want from a site, rather than what it can give them.
  • UX and the Art of Digital Appropriation | Brian Hoadley User experience is pervasive. It is ubiquitous. Companies and agencies need to step back and realise that the concept of user experience will mean change in the way their businesses operate. It’s too big to be owned by any one team, shoved down any one silo. It is too fundamentally important to leave to any one concept, methodology or team. To understand user experience is to create a fundamentally open and collaborative environment with a healthy exchange between and amongst users, businesses, agencies.
  • Data Won’t Solve Your Problems Fewer and fewer problems these days are technical. We have the data, and we even have the power to slice-and-dice it a million different ways. Never before have we been able to accumulate, manage, and analyze data to a greater depth than we can today. Here’s the problem: we don’t know what questions to ask of our data, nor what to do when he get an answer.
  • Financing efficiency | Blog Government needs to realise that more risk transfer doesn’t lead to better risk management. There are a variety of types of risk involved in each project. Government’s aim should be to transfer only those risks that it is within contractors’ powers to mitigate through innovation and efficient delivery.
  • Why I’m not on Google Plus – Charlie’s Diary Google are wrong about the root cause of online trolling and other forms of sociopathic behaviour. It’s nothing to do with anonymity. Rather, it’s to do with the evanescence of online identity. People who have long term online identities (regardless of whether they’re pseudonymous or not) tend to protect their reputations. Trolls, in contrast, use throw-away identities because it’s not a real identity to them: it’s a sock puppet they wave in the face of their victim to torment them. Forcing people to use their real name online won’t magically induce civility: the trolls don’t care. Identity, to them, is something that exists in the room with the big blue ceiling, away from the keyboard. Stuff in the glowing screen is imaginary and of no consequence.
  • The Difference Between UI and UX | Design Shack At the end of the day, that is all we get to leave the user with: a memory. As we all know, human memory is astounding but it’s imperfect. Every detail contributes to the ingredients of a good user experience, but when it all comes down to it, the user will remember products in somewhat skewed way. UX contains a much bigger picture than UI does but it still relies on the smallest details to drive it. This understanding is the most powerful asset anyone can have in product development.
  • Why It’s Good that the Internet Is Changing Our Brains – Technology – GOOD We all want to believe that as humans, we control our tools, not the other way around. Clark would argue the exact opposite is true, and for the better. After all, how can we say our brains are all we need to be our “real selves” when we have so much stored and invested in our outside technologies? Maybe we’re not losing our “selfhood” at all, but creating mega-selves. Perhaps we should be thinking of our presence on the internet, our phones, and our hard drives as equally important parts of us—really clever parts who can tell jokes in 140 characters or less.

Aphorism 56

8 September 2011

This is the futurist’s dilemma: Any believable prediction will be wrong. Any correct prediction will be unbelievable. Either way, a futurist can’t win. He is either dismissed or wrong.

Kevin Kelly

Cleaning up the user interface

6 September 2011

My dishwasher has a bit of whatever the white goods equivalent is of bling. It has a display panel on the front conveying mostly irrelevant information fairly inefficiently. I assume it is intended to communicate whizzy modernity; it certainly doesn’t communicate much useful information.  It slowly cycles through three screens, only one of which tells me when it will finish, which is the only thing I want to know. But the screen is there, glowing with potential.

The actual controls are tucked away inside the door. They are simple enough and include another display, but one which is much more limited, able to show three crude characters.  At the moment it is showing ‘E-24′. This is clearly not good, since the dishwasher has stopped mid-cycle with a puddle of dirty water at the bottom  But in what way it is not good, I am none the wiser.

Meanwhile, the big screen at the front is showing precisely nothing. It could so easily give me an E-24 related message: the first level translation when I found it took four words, the second level explanation of what to do about it took 75 more to give far more detail than I actually needed. Either would be well within the capabilities of the screen, neither is attempted.

No doubt the deep significance of E-24 is explained in the manual, but that’s just an assumption, since the manual is not there to check. So the obvious answer is to google for a solution. I know the manufacturer, I know the error, but it occurs to me that the model number might be relevant too. There is a helpful sticker giving me a phone number to ring, but no sign of a model number (and what’s the betting on the first question they would ask me if I were to ring the number?).  Too bad, time is running short, I am supposed to be somewhere else. And google comes through without it: a simple tweak, turn  off, turn on, off it happily goes.

The point of all this is the balance between the normal and the abnormal.  Someone has put a lot of thought into the information this machine gives out when it is in a normal state. That’s the state it is in almost all the time, so at first sight, that’s a sensible thing to do. I don’t know how often across the universe of dishwashers E-24 comes up, maybe it’s just once in a month of Sundays. But precisely because it is rare and unexpected, that’s when I need help understanding what it means and what I should do.

It’s terribly easy in designing any service to put disproportionate weight on the core process which lead to successful outcomes. Newly fashionable approaches, such as agile, can be understood as encouraging that, with an emphasis on delivering highest value first. The catch, of course, is that value is not necessarily the same as volume, but the two can easily be elided.

Of course it’s right to make sure that the core experience works well: it’s necessary, but it’s not sufficient. Developing the first, second and nth level support processes always seem to lag behind the process which might need supporting, rather than being seen as an intrinsic part of them. So edge conditions can be overlooked, but more subtly to the extent they are developed, they more easily fall into a more technocratic or expert user approach.

In this respect, dishwashers are like staircases:

When specialised users are in a position to specify a service to meet their needs, that is what they will do, even if they are a small minority of the overall user base.

I am sure that dishwasher repair people the length and breadth of the country can recite error codes in their sleep: E-24 is all they need to know. But they are a tiny minority of the people affected. Or as Brian Hoadley has put it in a recent blog post:

No Virginia… you are NOT the user. The user is the user. Someday, you may have the opportunity to be a user on a project – where you are not doing the design. But until that day, please, everyone involved in projects, work harder to get to know who your users are, and please, do involve them in the process. It can be really quite rewarding!

The screen on the front of my dishwasher remained resolutely blank. But if you too need help with E-24, the answer is out there.

Game over

3 September 2011

Some evenings on my way home from work, I play a small private game of chicken. Now I have played it for the last time.

If there are too many people at the bus stop – and no, I don’t know how many makes too many – that’s a sign that the gap between buses is longer than it should be. That means that when the bus does come, it will be more crowded than it should be. So far, so straightforward. But it also may well mean that there is a second bus not far behind the first. That’s reasonably likely partly because the second bus may be on time even if the first one isn’t, but partly because it is the iron law of buses that once one is delayed and fills up, the one behind will start to catch up.

So as the first bus arrives, I have a choice. I can get on the first bus with complete certainty, but less comfort, or I can wait for a possible second which may or may not be close behind and may or may not be relatively empty. Most people don’t make that choice: with rare exceptions, everybody else crowds on to the first bus. On good days, the second bus comes round the corner when the first bus has barely moved away from the stop. On bad days, there is a long wait for a bus which is eventually just as crowded as the first.

It’s a gamble, but I have learned to be canny with the odds, and mostly I win the game. Mostly.

The game only exists because of an information deficit. TfL knows where all its buses are, but the rest of us can only guess, except at the fairly small proportion of stops which have dot matrix indicators. The value of making that information more widely available has been self-evident for years. Just over two years ago, a team at Young Rewired State calling themselves TfHell won the Public Strategist award for “the service which most obviously ought to exist, for which there is no good reason that it does not exist and which now just needs to happen”.

But alas it did not just happen – until now.

I played my game for the last time this evening. Or rather today the game played me: the first bus was so full that it went straight past without stopping, so I didn’t get to choose. But within a couple of minutes of finding a seat on the second bus I found that the world had changed.

[blackbirdpie url=""]

The effects should be interesting. Right now, it may not do much more than give me an edge in my decision making. If it were to get widely used, though, it should in theory produce an extraordinary feedback loop. If enough people wait for the second bus, knowing it is only just behind, rather than scrambling to get on the first, buses will get more regular and less crowded. A whole of host of individuals doing nothing other than making self-interested decisions based on a small but critical piece of information they don’t currently have, would improve the efficiency of the bus network.

The seminal work of observations of commuter behaviour, Notes from the Overground, is sadly very long out of print. Its author distinguished goats who seize and act on information to make their journey easier from sheep who do not. It’s a reasonable bet that even with better information, most people will still crowd on to the first bus (that is, after all, what happens now with tube trains, with the information available to everyone). That means that buses will continue to bunch up. But I will know that there is a second one, and that’s the one I will be on.

The web page says firmly that this is a test service – and indeed as a I write it is not working. Perhaps these last few hours have been all that we are to get for a while, and I will have to keep playing the game a while longer. In any case, there needs to be less to it than there is:  web pages designed for full screen browsers are a great way of consuming many kinds of content, but waiting times at bus stops is not one of them.

The first version of this has come much later than it should have done.  Let’s hope the second is not very far behind.

Update – 7 September

I have since discovered that there are three versions:

Desktop – with interactive maps

Text/Accessible – with static maps

Mobile – which is pure text with no maps.

Slightly confusingly, the desktop version has links to what it calls a text version, but is actually the sub-domain called ‘accessible’. The version which is actually text-based is not linked from either of the other two. Slightly more confusingly, although all three versions allow you to record a set of ‘My Stops’ for quick reference, the cookies which hold the data are held separately in the three sub-domains and so are invisible to each other (at least I think that is what’s going on; mine keep disappearing, even though I have given them an exemption from my standard cookie setting of eviscerate).

There is a page on the TfL site which explains a bit about the project – and it’s worth saying that this appears to be an unintentionally public beta at the moment, which deserves some latitude.

Independently of TfL, James Darling has produced a super-minimalist version which shows only times in dot-matrix effect red, and for hardier souls, Adrian Short has a command line version.

Picture by photosteve101, licensed under creative commons



Aphorism 55

30 August 2011

Archimedes had taken baths before.

Quoted by Alistair Croll

Interesting elsewhere – 19 August 2011

19 August 2011

Things which caught my eye elsewhere on the web

  • USENIX 2011 Keynote: Network Security in the Medium Term, 2061-2561 AD – Charlie’s Diary It’s nearly impossible to underestimate the political significance of information security on the internet of the future. Rather than our credentials and secrets being at risk – our credit card accounts and access to our email – our actual memories and sense of self may be vulnerable to tampering by a sufficiently deft attacker. From being an afterthought or a luxury – relevant only to the tiny fraction of people with accounts on time-sharing systems in the 1970s – security is pushed down the pyramid of needs until it’s important to all of us. Because it’s no longer about our property, physical or intellectual, or about authentication: it’s about our actual identity as physical human beings.
  • The Ministry of Lorem | Helpful Technology So I knocked up Ministry of Lorem. It grabs the latest press releases from about a dozen central government departments who publish usable RSS feeds, cleans up the first line a bit, randomises the order, and generates as many paragraphs of text as you need. Oddly mesmerising stuff:
  • A tale of two networks | Curiouscatherine’s Blog The problem here is so obviously not the technology –to say so is to take a technological determinist view of the world that ignores the fact that we have been on a path to a more networked society every since the telegraph enabled us to reach across the planet. You can no more remove the networked behaviour at this point than you can stop people talking on street corners.
  • The invisible website by Michele Ide-Smith I found there are websites and web applications I admire from a user experience perspective, because as a user I enjoy using and interacting with them. They minimise frustration and maximise the experience of finding information, carrying out particular tasks or sharing and communicating with other people.
  • HMRC – not very nice but a lot more efficient | Flip Chart Fairy Tales People won’t like it. Most no longer believe the more-for-less spin but a lot still think they will get the same for less. They will be in for a shock. Performance and productivity are not the same thing. The public services of the future will, hopefully, be more cost-effective but, in becoming so, they will won’t give us as much as we have been used to. In some cases, cheap may well mean nasty too.
  • Wicked (1) – Charlie’s Diary It is not the case that wicked problems are simply problems that have been incompletely analyzed; there really is no ‘right’ formulation and no ‘right’ answer. These are problems that cannot be engineered. The anger of many of my acquaintances seems to stem from the erroneous perception that they could be solved this way only those damned republicans/democrats/liberals/ conservatives/tree-huggers/industrialists/true believers/ denialists didn’t keep muddying the waters. Because many people aren’t aware that there are wicked problem they experience the failure to solve major complex world issues as the failure of some particular group to understand ‘the real situation.’
  • Difference & Relationship Between Usability & User Experience | Usability Geek In terms of a web site, the aim of usability is to make that web site easy to use whilst the aim of user experience is to make the user happy before, during and after using that web site. Thus, usability relates to the ease with which users can achieve their goals while interacting with a web site while user experience is concerned with the way users perceive their interaction with that web site
  • John Kay – Kipling’s game theory lessons for Greece In the dollar bill auction, one party eventually scores a pyrrhic victory and takes possession of the dollar bill. Both parties lose, but the smaller loser is the person who sticks out longest. That is not usually the rational player.
  • Contrast Rebellion – to hell with low-contrast fonts! Clearly, aesthetics are important but aren’t the ultimate goal of design. And often poor readability doesn’t get noticed during the design process, as we are not like our users. We don’t read the texts as a visitor does.
  • Five Popular Web Strategies That Don’t Work | UX Magazine While usability is a must for long-term success, it’s really just table stakes. If your websites and products aren’t useful as well as usable, then all the usability in the world won’t help you.
  • What’s the next challenge for Open Government data? | Emma Mulqueeny Forget the data. Find a way to enable these revolutionary ideas, apps, websites and widgets that save time, money and mind-numbing frustration from those who have to engage with government. Do that, and only that.
  • Man walks into a column, no.29: Change « arbitrary constant My point is that no amount of cajoling or persuasion will get service users over the natural resistance to change if they can’t experience the benefits of a redesigned service themselves (and after all, we know most politicians are liars, as I’ve blogged before: why trust them?). It’s a version of the ‘if you build it, they will come’ argument. We may hate change, but it’s amazing how quickly and radically we do change if we find something better than we had before.
  • Pervasive UX – Who is Responsible? – Digital Optimist As businesses and organisations offer more information and functionality through more channels and in more places, it is imperative that someone considers who all of this was being done for to begin with. If the output, due to internal disagreement, is already a compromise, then the experience for the individual can only be fragmented.
  • Stumbling and Mumbling: Rebekah Brooks & modern management Could it be that remote control managers function much as the gods did in ancient times. They get blame when things go wrong and praise when they go right, but in fact have no power at all, except that which ignorant people impute to them? They are, technically, redundant and are sustained in their lucrative positions only by superstition and ideology.
  • Our Responsibilities as Citizens | Involve As we discussed the issue, I began to reflect that social media is changing the world far faster than I had understood; the very power of social media to give our opinions a platform also constrains us. With the power of social media comes a level of responsibility that we, who are used to viewing ourselves as private citizens, are only just beginning to understand. We are increasingly public citizens with all the implications that this entails.

Transparent government

1 August 2011

I have read three blog posts in the last few days which together strike me as indicators of a welcome trend.

The first two were both from the new Government Digital Service blog, the first from Mike Bracken on his first few weeks in government as Director of Digital. It’s a very upbeat assessment, but the important (and still remarkable thing) is that it is there at all. The second is from Tom Loosemore, reflecting on the experiment, in which he started by setting out the top ten problems reported by users (and giving quick responses to them), and only then moving on to praise and ideas. Having the self-confidence to be open and balanced like that shouldn’t be remarkable, but it is, and every example helps make it more normal.

The third was very different: an account by Liz Azyan of the importance attached to user testing in the redevelopment of Camden’s website. It is unabashed and powerful argument, reinforced by a very ungovernmental summary, boiling the whole thing down to four speech bubbles. More importantly, it is part of a very conscious strategy of openness. Reading Liz’s post after  Mike’s and Tom’s triggered the thought that there is a pattern here which is worth noting and encouraging.

There is more to transparent government than open data. Understanding how things are done and what decisions were made about doing them can be at least as important. That understanding is not easy to acquire from the inside and is harder still to get from the outside. That’s partly because of an inherent limitation of communication, but it is also because history is written by those who write history.

Two years ago, I wrote a taxonomy of government bloggers, partly to make the point that it was heavily skewed to certain groups and roles. With the long hindsight of those two years, it now seems odd not to have had a category of those who were making change happen, and critically who were being open about the change they were part of. There were people among my examples who did that, but that wasn’t quite the reason they were there. I opened the description of my first category, those whose job it is by saying:

There is a vibrant online community among those whose job is to make the government a place of vibrant online communities.

That is still true. The good news is that there are now signs of a vibrant community among those whose job is to change the way government works for the better. The challenge is that that community is still predominantly drawn from those for whom digital is the substance of their work as well as a medium for communicating about it. That may become less important as digital increasingly does become the default, but there is still plenty of scope and much value to be had in widening the voices of government.

Petitioning the elephant

29 July 2011

A new e-petitions site for government was launched yesterday. It is clean, simple and elegant, with clear government branding consistent with other cross-government sites.  So far so good. But managing a cross government service is a tricky business, for reasons I have explored before. Government is a veneer that sits above departments, and like any veneer, can easily crack if the underlying structure is not firmly held together. If you want to understand why government online services so often feel as though they are not quite all that they could be, this is an example worth reflecting on.

An eager e-petitioner clicks the button to start the process and finds themselves with a simple form to complete. The first task is to give the petition a title.  Pretty straightforward. The second is to identify the ‘Department that looks after your issue’. That’s a poser. There is a drop down list. There is a link to a page which explains which department does what. But the list is of ministerial departments and the help page gives little more than mission statements. Many of the bits of government which people have at least some understanding of don’t appear at all – there is no HMRC, no DVLA, no NHS, no Jobcentre Plus. Might a petition be appropriately directed to the Scotland Office, or should it go to the Scottish Government instead?

So I wrote a slightly frustrated tweet:

[blackbirdpie url="!/pubstrat/status/96851442098376704"]

And got a response I have a degree of sympathy with:

[blackbirdpie url="!/marxculture/status/96852213485412352"]

There are parts of any interaction where the user is expert, and there are parts where the provider is expert.  One of the keys to good service design is to ensure as far as possible that each does the right bit. Asking petitioners to map petition subjects to government departments fails that test.

So why does it work like that? I have no inside information, so can only guess, but the answer might come from asking what might have to be different to make the problem go away.

At a first glance, there are two basic ways that might be done. The first is to attempt to use the petition itself to answer the question. A little bit of keyword analysis, a touch of natural language processing and the job is done.  That would be in the spirit of the approach, and the suggested destination could then be reflected back to the petitioner for a human check.  The second would be not to bother at all at this stage:  just send them all to somebody whose job it is to do the routing. There is also a third which wouldn’t make it go away, but might reduce its impact, simply to ask the question later and avoid its jarring position between petition title and petition description.

The problem with the first is that it makes a simple thing much more complicated.  Essentially, the e-petitions site is a simple web form.  That’s a feature, not a limitation: the project was designed from the outset to test and demonstrate an agile approach, where finding ways of making things simple is critical to finding ways of making them happen at all.  Adding on the kind of analytical tools to add intelligence to the routing might have slowed things down a bit – but might well have stopped progress altogether.

The problem with the second is that it risks introducing a cascade of consequential problems. There is nobody whose job it is to do this kind of routing, because this kind of routing is not done. So a role would need to be created, made part of a team, attached to an organisation chart, found a desk, found funding – not just for this year but for next year, and the year after that. There is nothing more to that, in a sense, than the issues any project has moving its creation into live running, but in a different way this too might have slowed things to paralysis (though reading the site’s terms and conditions shows pretty clearly, that some of that is needed anyway: if ‘it will usually take up to seven days from the time an e-petition is submitted for it to appear on the website’, the back end is human – and if one of the humans could reflect  on the use of ‘usually’ and ‘up to’ in that sentence, the world could easily be made a better place).

All that perhaps boils down to saying that it is is better to have something now rather than something potentially better in a future which may never be reached. That’s not a bad argument, but in this case at least, I think it is wrong.

It is increasingly, and rightly, said (including by me) that there is no such thing as an IT project:  there are only business projects with IT components. The corollary of that is that there is no such thing as an IT solution, there are only business solutions supported by IT. A solution is not a solution if it does not support the right overall experience and outcome. The e-petitions site is partly about e-petitions, partly about showing what a skunkworks can do, and partly about pointing to the future of government services. It has done the first two admirably. I am not sure that it has done the third.

Update 30 July

I have just stumbled over a small piece of minor irony in the newly integrated Government Digital Service blog:

Directgov was launched in April 2004 to allow people to access public services and government information in a single place. We know that people care little about the structures of government and do not want to go searching around the internet so the original aim of Directgov was to join up services online.

Although visually unrelated, the e-petitions site is formally part of Directgov.