CMT4240: Designing Interactive Systems
Identifying Users' Domain Knowledge
After this session you will be able to
Appreciate the importance of domain knowledge
Describe a range of techniques for gathering data about users' domain knowledge
Discuss the strengths and weaknesses of each
Be in a position to choose techniques appropriate for a particular project
To improve usability need to go beyond the interface…
Need to understand the context our designs will be used in
As well as understanding context, need to know how the user acts in context
Domain knowledge: knowledge about the environment and task
Knowledge people have about context
What to people know about?
"facts" about the world and their job
facts about the devices and systems they use
ways of carrying out tasks
Make more clearly the distinction between domain and device
Knowledge elicitation difficulties
Level of operation
experts often do things rapidly and unconsciously
knowledge the person doesn't know they have; common knowledge
taken for granted; sometimes better to look at beginners
between "explicit" and "tacit" knowledge
Identifying domain knowledge
Questions to ask
purpose of investigation?
users of the to-be-designed system?
Design an interactive information system to help tourists and visitors to London
What kinds of knowledge might users possess?
How can this be useful in design?
How can we go about finding out?
Possible research methods
Many different techniques for analysing interaction
Log files, Workshops, Scenarios, …..
Different methods yield different information about users' knowledge
To find out what people what know about their work - ask them!
Can be highly flexible and responsive
Can be more or less structured
Some planning in advance necessary
Interviews - 2
May be carried out “In context”
Focus of interviews:
Typically involves audio recording for subsequent transcription and analysis
Good for knowledge people are able to articulate
less good for expert, tacit knowledge
Often used for evaluating systems, finding out about attitudes, preferences, surveying user population, …
Fixed pre-planned set of questions
Not flexible or responsive
What kinds of information are elicited using questionnaires?
Questionnaires - 2
Allows comparisons and analysis
Cross check and validate other data (interviews)
When are they more useful than interviews?
Generally poor way of eliciting users' knowledge
People often not good at explaining what they know, so
Watch what people actually do in their natural environment
Often as a participant
Build up a rich understanding of knowledge used, social organisation, context, …
Observation - 2
Tends to be in-depth, and looks at only a small number of users
Good or identifying what people do
Identifying why and what knowledge is used typically time consuming:
Ethnographic studies may take months, involve "immersion" in field
"Discount" techniques require less time. E.G., Contextual inquiry
Allows analyst to identify when and how in an interaction user’s knowledge is used
Involves observing user activity, making notes or video / audio record
“Think aloud protocol”, “retrospective protocol”
Analyst may prompt user
Recordings are transcribed and coded using a specialised coding scheme
Not applicable in all situations
Questions and prompts:
How do you….?
What are you trying to do?
What will happen if….?
What has the computer done now?
What does this message mean?What did you expect to happen?
Involves formulating hypothesis
Carried out under carefully controlled conditions
Useful for comparing design alternatives (for example) and usability studies
Not useful for exploring users’ domain knowledge
Rich source of data; Relatively quick to administer; Flexible; Focus on what people can talk about
Data may be easy to process; Quick to administer; Rigid; Allow large samples and comparison; Often poor for knowledge elicitation
Comparison - 2
Rich data; Time consuming to elicit users’ knowledge; Focus on what people do; Analysis may be complex
Analysis may be complex; Requires some skill of researcher; Requires users to talk and act; Useful for eliciting users' knowledge
Techniques have different strengths & weaknesses and can explore different aspects of domain knowledge
Using several techniques can help to build up a richer picture
Verify findings using several techniques and different samples
Case Study: Emergency Ambulance Dispatch
Project to understand decision making in ambulance dispatch and critique proposals for new technology
Large and complex system
many people with different jobs
Used a combination of techniques to analyse from different angles and validate findings
Techniques Used in the Study
Open, informal interviews with senior management for background
Notes and video record of several sessions
Critical incident analysis
Controllers possess rich knowledge
About state of the world
About previous incidents & clinical decision making
Knowledge about computer systems also required
Fit between technology and domain tasks?
Context and domain knowledge is important
Can inform design
Many techniques exist to analyse knowledge users possess
Can be used together to provide richer data and cross check