Dr Paul Curzon

Human Error Projects

See the Space Invaders game based on a previous project in this area This led to a research paper for the students concerned, when combined with another similar project using a Driving game, at the HCI 2006 conference. It won a best paper prize.

J Back, WL Cheng, R Dann, P Curzon and A Blandford (2006) "Does being motivated to avoid procedural errors influence their systematicity? People and Computers XX - Engage, Proceedings of HCI 2006, pp151-158, Springer.

The project suggestions in this section are connected to the EPSRC funded CHI+MED on the design of medical devices, and its predecessor the HUM Research Project (see: the project page ).

They mainly involve doing experimental investigations of the causes behind people making systematic errors. Note that these are not easy projects as they require good programming (or at least high fidelity prototyping) skills and an ability to develop good experimental design and execution skills including the statistical analysis of the results. However students who are up to those challenges usually get very good project grades.

When designing the project it is important that you ensure it demonstrates your Computer Science skills and is not just a pure psychology experiment. That might involve producing a complex program as the basis of the experiments including reusable infrastructure to record the results of experiments automatically (errors made, time taken, etc for each participant). It might concern different interface design - focussing on comparisons different interface tequniques for avoiding the error.

Most involve developing an interactive system. This system might be a game, a puzzle, a simulation of a medical device, a central heating controller, part of an aircraft cockpit etc. An important point is that it must be sufficiently realistic and engaging so that participants take it seriously enough.

The examples below are just ideas. To have a project you need to:

  • a systematic error people (possibly you) make often, eg missing first initialisation steps of a sequence, missing last tidying up steps
  • think of a research question related to human-computer systems concerning it, its causes or how to design to avoid the problem,
  • work out a paradigm (a game, a puzzle, etc) ie what you will build to help explore the question
  • design an experimental set up that will answer the question. The best way is to keep this simple and eg have two groups each using a version of your program that differs in one way only - the thing you think will have an effect on errors happening.

The question must be novel to some extent, or be answered in a novel way. However the papers below may get you started.

Further Reading: The following are experiments along these lines done previously, mainly concerned with one particular kind of error.

  • M. Byrne and S Bovair (1997) A working memory model of a common procedural error. Cognitive Science 21(1):31-61.
  • Ian Mortenson (2004) An investigation of working memory areas: post-completion errors and the implications for HCI. Master's Dissertation. Middlesex University Interaction Design Centre Technical Report: IDC-TR-2004-005. February 2004 pdf (51 pages) pdf
  • Chung, P. H., & Byrne, M. D. (2004). Visual cues to reduce errors in a routine procedural task. Proceedings of the Twenty-Sixth Annual Conference of the Cognitive Science Society. pdf

Paradigms

In these projects you create a program that can then be used in the experiments. Usually you would make two versions of the program with one difference. It is the effect of that difference on error rates that you investigate. The program could be doing a variety of different things. Some ideas are below.

Microworlds and simulations

You could create a microworld. This is a little like a game but is simulating some real situation. For example it could be a simulation of a drug delivery device. You then build a scenario around it that the users are nurses setting drug doses for different patients. You can then investigate if different designs or other factors effect whether the nurses make errors (that might lead to overdoses). Other possibilities used in the past for this kind of experiment have been air traffic coontrol scenarios, fire engine dispatch, reading gas meters, even donut making machines, ... These have the advantage of being partly realistic - looking at situations where errors really do matter.

Video games

The program might be a video game - any of the classics like Pac-man, shoot-em games, card games etc. Games can have the advantage that people can get immersed in the game. The errors may be in side tasks (like remembering to save your score at the end) or something critical within the game where failure means you lose points. You could develop an audio game and investigate if that makes a difference.

Puzzles like Sudoku and Kakuro

You can use logic puzzles like Sudoku and Kakuro to investigate whether people make mistakes when they are having to think hard about a problem. Are people more likely to forget to press save buttons when doing harder puzzles? Can cues help, etc. Simpler puzzles like "crossing the river" type puzzles could also be used to investigate whether interface cues make a difference.

Kinds of errors

There are a whole range of errors to investigate. A few classic ones are described below. The key thing is that the person should have known what to do but just not done it.

Initialization errors

With my microwave I often forget to press the reset button before I enter the length of time the food is to be cooked. People sometimes forget to take the handbrake off on the car before moving off. They forget to click on a window, before starting to type into it (eg a password window). These are all initialisation errors.

There is some evidence from previous experiments (involving operating online Donut machines) that particular kinds of initialisation errors are very common with some interfaces. The problem seems to relate to having a series of actions A, B, C .. but where each needs to be initialissed - ie move into an appropriate mode to do the following subtask, so the person must do InitA, A, InitB B, InitC, C. What seems to happen is that people regularly forget to do the initA action - the first initialisation, but have no problem remembering the later initialisation actions (InitB and initC). It seems to be something to do with them being device specific actions rather than ones central to the task. This project would be to create software (eg a game of some kind) and run experiments using them to find out more about this problem. It might for example involve creating an adventure game where people move around a series of rooms on the screen completing tasks dependent on the room.

Habit based errors

A common reasons for humans making systematic errors when using computer systems is due to habitual behaviour. If a person has repeatedly performed some series of actions then they are liable to perform the same sequence again when they are no longer appropriate. For example, perhaps the same series of three OK clicks in pop-up windows are needed to perform a task. However, if a different action is then needed where the first two actions are the same but for the last one, OK is the wrong action, then people are liable to do the wrong thing. If systems are designed so that such situations can occur then arguably the design is flawed as it encourages human error. In safety critical systems such as aircraft cockpits it could be fatal.

The aim of this project is to investigate the causes of habitual errors and how they can be avoided. The project will involve developing test programs around which user studies can be based. They might, for example, simulate a simple interactive device design (perhaps a toy airline cockpit or windows application) with a habitual error design fault. A user study will then be carried out to investigate the circumstances in which users make such errors with variations of the interactive device. Device designs without the problem should also be developed and shown to be free of the problem. develop a test program to allow an investigation

Specific questions might be about how long it takes to learn a habit, and/or then unlearn it for different kinds of interfaces - eg is it harder to unlearn if clicking buttons than clicking in entries in a drop-down menu.

Post-completion errors.

Post-completion errors are human memory errors that can be avoided by good system design. People make them when undertaking a task where they achieve their main goal, but then forget to do other completion tasks that arose during the interaction. For example, a person filling their car with petrol might forget to replace the petrol cap, or when cooking they might forget to turn off the gas. Early cash machines had this problem: users would take their cash but forget their card. Modern cash points are designed so that this no longer occurs.

As with the other errors, there is lots of scope to design and carry out user studies to investigate factors that cause people to people make such errors. The test program developed could be based around several different task paradigms. For example it might involve a dungeons and dragons style game, some kind of puzzle, a driving simulator...or something more appropriate. Part of the project will involve determining a paradigm suitable for the factors under test.

Communication Goal based errors

Users have goals in using an interactive system, and in achieving them have sub-goals they know they must do to complete the task. FOr example in using a cash machine you know you must provide your card and give a PIN. Goals can be of different kinds:

  • Ones where you know you have to communicate information
  • Ones where you must provide an object
  • Ones where you must perform an action
  • ones where you must solve a problem
Investigate how errors based on these different kinds of goals differ. Are there kinds of errors that are more common with some but not other kinds of goals.

Ways to reduce error and causes to investigate

There are lots of different factors that might be investigated that might affect error rates, such as the order actions must be done, visibility of the system state, percieved importance to the person, distractions, social context, etc. For example you might investigate how obvious a reminder must be to guarantee it is not ignored, or how close to the time when the completion action is needed (research has shown that early reminders d not work and suggest it must be just in time - but how just in time must it be?).

Can expert games players learn not to make the mistake at high skill levels?

Audio versus Textual loads

People are more likely to make some errors when they have a lot of extra information to filter that is not directly part of the main task. Does it make a difference if the load someone is under is listening to information or watching it on a ticker tape?

New versions of devices

Nurses have to deal with lots of similar medical devices that look the same but behave slightly diferently, because a manufacturer releases a new version eg on one, when you press the units button it might overflow to the 10s (8,9,10) and on the next it goes back to zero (8,9,0). What situations does this lead to problems and how can they be avoided?

Strategies for avoiding error in a game situation

Investigate strategies game players use to avoid making errors. For example anecdotal evidence from the the Space Invaders game experiments suggested that players used a variety of strategies (like always moving to the side at the point where they would otherwsie make an error so they had less to do. Design a sufficiently rich game with embedded error opportunity and run experiments to test how people avoid making errors. This could also lead into an investigation of how to design interfaces to help people avoid problems.

Explicit cues

In the spaceinvaders experiment it looks as though the subtle cue in the image of the spaceship changing is enough to help avoid the error. What would happen if there was a more subtle cue, or a similar cue in a different part of the screen where the person's attention wasn't focussed. This project will involve developing a new game to test such issues.

The effects of double errors

Anecdotal evidence has suggested that after a person has made one slip error they are more likely to make a subsequent one (forgetting to do something) if they think the first error was their fault: they know they should have known and avoided it. This would involve setting up a suitable game or other task software and running experiments that involve a series of trials to look to see if this is the case and explore the causes - for example is it that they are flustered. For example they might be trying to unlock chests with keys but use the wrong key because it is nearest even though it is clearly the wrong one. The game could be rigged in some way to make the first error highly likely (eg no cues at all - it is all down to remembering)

The effect of boredom or pressure on error

When people are very bored and also when they are very preassured their performance deteriorates (this result dates back to a 1920 paper by Yerkes-Dodson. This project would explore this and in particular its effect on people making slip errors. In the experiment there would need to be a mix of slack times and very active times for the participant with error opportunities in both. For example the game might be an exploration game around rooms with things like doors needing shutting but tense things happening in some rooms but not for other long periods. When do people forget to shut the doors?

The use of greyed out keys

Investigate what interface techniques are best at preventing a person clicking a button. Possibilities are

  • to leave the button unchanged
  • to grey it out (to varying degrees)
  • different shapes
  • different colours

Create a game or puzzle in different versions, where there is some reason why a person might believe they could usefully press a button at some time when it has no effect. Investigate the differences in error rates for different methods of highlighting it is unavailable.

One possibility would be to engineer a situation where pressing the button becomes a habit due eg to having a moded system. That would make the number of errors high and allow to see if any technique reduces it.

An executable model of cognitively plausible human behaviour

It is possible to take some of the basic principles behind the way people act in terms of their goals and knowledge of tasks and devices and write a program that simulates this behavious. Such an executable model could be used to test interactive devices (eg cash machines, ticket macines, video recorders, etc) for design flaws that lead to systematic human errors being made. Changes to the design of the system under test could then lead to the specific human errors being eradicated.

The aim of this project is to develop such a program and to evaluate its use by developing also simulations of simple devices, both with and without design flaws to test it against. A test harness for the device could also be developed to help run tests.

Is it safe to use voice-based personal computers while driving?

Currently, the law in the UK is that people can use telephones while driving, provided that they are hands-free. Increasingly phones are merging with personal computers with applications such as having emails read out via the phone. However, studies of mental workload suggest that even simple phone conversations might not always be safe. Many competent drivers will quite happily chat to a passenger while driving, but may go quiet if road conditions get stressful, as they concentrate on the primary task (driving safely). There is one important difference between the two situations: A passenger shares knowledge of the driving conditions with the driver, whereas someone at the other end of the phone does not. There are clearly at least two factors that might affect how safe it is to use a phone while driving: the traffic conditions, and awareness of the other speaker of the situation. There may be other factors, such as how well the two conversation participants know each other. The proposed project is to investigate this phenomenon, but in a safe way (no real cars!) by creating and testing an analogous situation.

Method:

  1. Design and implement a challenging game with periods of low activity and periods of intense activity that come unpredictably - analogous to driving.
  2. Collect think-aloud protocols of users playing the game under various conditions:
    1. single-user (thinking aloud about some topic such as future holiday plans)
    2. co-present companion with whom the challenge is to maintain a dialogue about something - e.g. to plan a detailed holiday itinerary
    3. remote companion, same task
  3. Compare performance under the various conditions.
Other projects might consider the kind of information being presented. For example comparing route information as from a navigation system, personal emails and work emails read out, listening to a news report etc.

Calibrated Games based on systematic human error

The aim of this project is to develop a series of difficulty levels of a game that are based on systematic human error induced by interface features. See (and play) the Space Invaders game . It includes an induced error where the user forgets to switch on the gun after it has been switched off. This could be made harder for example by switching off the visual cue in that the ship icon changes. Design a game with levels based on, as well as faster pace etc such error related design changes researched from the literature and validated by your experiments with the game.