The intelligence of the crowd

Twenty years ago, I had a problem to solve. A museum was creating a database of the objects it held and needed to be able to produce reports in catalogue number order. The difficulty was that different numbering systems had been used at different times and catalogues with different numbering systems had been amalgamated without changing the separate numbering systems. The database needed to include objects acquired and given a number over a hundred years ago, and objects acquired yesterday. Some numbers had the year of acquisition as a prefix, some had it as a suffix, and some didn’t have it at all. Some numbers were just numbers, some numbers also had some letters. Some numbers had hyphens in them, some didn’t; hyphens appeared at different points in different kinds of numbers, and meant different things depending on where they appeared, and so it went on.

For the curators, none of this was a problem. Given an arbitrary list of catalogue numbers, they could always tell you what the ‘right’ order was. They were completely consistent about it and their approach was perfectly logical – but it had very little to do with the ASCII ordering the database program liked best.

So my problem was to write a routine which would parse a catalogue number entered in free text form, work out which of the various possible sequences it belonged to, format it to look right to the curators, use that version whenever the catalogue number was displayed, create a separate version of the number across several fields which could be used to generate an index which then came out in the order the curators expected, all without requiring them to enter the numbers in the first place other than in the way they had always written them.

It took me a day or two of thinking and a few more days of coding and tweaking, but I got there. I had no training or experience of this kind of thing, and had had to work it all out from first principles, so I was quite impressed with myself. The curators weren’t impressed in the slightest: the database just behaved as they expected it to.

Twenty years later I have a different problem. I am confronting the wisdom of many crowds distilled into the departmental expenses system. It has long been the mantra of Oracle and SAP that because their installed base is so vast, their software represents the best way of doing things, distilled from the experience of millions of user interactions. The organisations which use their software should therefore adapt their processes to fit the software rather than have the temerity even to consider adapting the software to fit their processes, as the software embodies the best possible way of doing things. Government departments don’t take that kind of challenge lying down, of course, so it doesn’t always turn out that way in practice, but the principle is clear enough.

So let’s start by entering the time into a time field. Simple:

0600

Or, perhaps, not so simple:

Please enter the time value 0600 in the correct format HH24:MI

Let’s try putting in the date:

13-05 is not a valid date.  Please re-enter.

OK, let’s try it a different way:

130508 is not a valid date.  Please re-enter.

There are things humans are good at, and other things which computers are good at. Putting simple data elements into standard formats is, without a shadow of doubt, something for computers, not humans. What’s particularly irritating about these error messages is that they are half way to solving themselves. It would take so little to go from saying that 0600 should be in the format hh:mm to simply putting it in that format.

So the interesting question is why there is no attempt to be more helpful. I don’t know the answer to that, so I can only guess. My guess, though, is that the underlying reason is that their users are not their customers. I can’t walk away from this system. It wasn’t chosen because it meets my needs.

That’s all a salutary reminder that the same is true for the systems we impose on our customers, which makes it doubly important that we take the trouble to establish what they need and how the experience will work for them.

But enough of this distraction, back to expenses. Or not:

Unexpected URL parameters have been detected and will be ignored.

Which I think means, you haven’t been using this for a while, so you have been logged out and will have to log in again. But that isn’t quite what it has managed to say.