Ethnographic research methods like contextual inquiry were used to understand user experiences of the university library website. Contextual interviews were conducted with students, faculty, and staff, followed by interpretation sessions to analyze the data. This involved creating sequence models of user tasks, affinity diagrams to group themes, and personas. The goal was to gain insights into how users work in order to design services and a website that better meet their needs. Challenges included the time and resources required, but advantages were an in-depth understanding of users and their research processes to inform improvements.
Contextual Inquiry: How Ethnographic Research can Impact the UX of Your Website
1. How Ethnographic Research Can
Impact the UX of your Website
Rachel Vacek, Head of Web Services
LITA Forum, Minneapolis, MN
November 14, 2015
@vacekrae
4. Once you invite [ethnographic] practices into the
the everyday way of doing things, it can be
institutionally transformative. It takes time. It is
inexact at times. It requires reflection, the
backing away from assumptions, it involves
being uncomfortable with what is revealed.
- Dr. Donna Lanclos,
Anthropologist,
UNC Charlotte
http://www.donnalanclos.com/?p=305
9. So how does ethnographic
research fit in with UX?
10. See the experience from user’s perspective
Appreciate significance of cultural differences
Understand motivations behind actions
Observe hidden behaviors
Learn routines in doing research
See how the user recovers from problems
Embrace user’s individual experience
11. …and fit in with libraries?
Better understand our users
Apply gradual improvements
Tailor services appropriately
Adapt to changing demands
Make evidence-based decisions
Expand the knowledge of our colleagues and
profession
18. Our research
Web and click analytics
Heat maps
Qualitative and quantitative content audit
Literature review
Review of web design and development
trends
Stakeholder focus groups
Usability benchmark
Competitive review
19. We thought that gathering
and analyzing user data
together to get
a shared understanding
of the users’ contexts
would be valuable.
23. It’s a user-centered design (UCD)
ethnographic research method
It’s a series of structured, in-depth user
interviews
The interviewer asks the user to perform
common tasks that he/she would normally
do
24. Human computer interaction
engineers created the
methodology in the late 1980s.
It’s based on theories from
several disciplines, including
anthropology, psychology and
design.
Contextual inquiries have rarely
been documented through
formal scholarly communication.
27. Unlike web analytics, we can understand the
intent behind their actions. If it’s unclear, we can
ask them why they did something a certain way.
28. Unlike surveys, we can interact with the users
and observe what they are doing. This is much
more accurate than self-reporting.
29. While surveys can be excellent tools for many questions,
and are attractive for LIS professionals because they can
reach many people economically, effective surveys
require knowledge of survey design and validation,
sampling methods, quantitative (and often qualitative)
data analysis, and other skills that require formal training
many LIS professionals do not possess. … Without
rigorous survey design and validation, data can lead to
results that are invalid, misleading, or simply not
meaningful to answer the question at hand.
- From “#DitchTheSurvey: Expanding
Methodological Diversity in LIS Research”
http://www.inthelibrarywiththeleadpipe.org/2015/ditchthesurvey-expanding-methodological-diversity-in-lis-research/
30. Unlike usability testing, it’s not under contrived
conditions in a laboratory-like setting. It’s what
they would really do in real life, not tasks we
designed to test specific parts of the system.
31. Unlike focus groups, an individual user is able
to talk about and show us in detail the way
he/she does things without the influence of
others.
32. …relying strictly on what students tell us in
focus groups is potentially incomplete … focus
group participants may share only what they
think we want to hear or they may fail to
accurately describe their library use. Listening is
important, but observation can yield unexpected
revelations.
- Stephen Bell, From the Bell Tower
column, Library Journal
http://lj.libraryjournal.com/2015/06/opinion/steven-bell/not-liking-what-users-have-to-say-listen-anyway-from-the-bell-tower/#_
34. The open-ended nature of the
interaction makes it possible to reveal
tacit knowledge.
The information produced by contextual
inquiry is highly reliable and highly
detailed.
The technique is very flexible as you
can conduct interviews anywhere your
users are.
https://en.wikipedia.org/wiki/Contextual_inquiry
35. Users can show us
what they do
rather than tell us,
and we can better
understand why
they do it.
37. Contextual Inquiry
Interpretation Sessions
Sequence Modeling
Affinity Diagramming
Visioning
Storyboarding
User Environment Design
Persona Development
Interaction and Visual Design
Paper Prototypes and Interviews
Product and System Requirements
Talk with users, capture key
issues, understand as a team
what is important to users, and
consolidate the data
Set a direction based on what you
know about your users
Inspired by https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/contextual-design
Design a system to support all the
research
Iterate designs and systems with
users
38. Contextual Inquiry
Interpretation Sessions
Sequence Modeling
Affinity Diagramming
Visioning
Persona Development
Set a direction based on what you
know about your users
Inspired by https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/contextual-design
Talk with users, capture key
issues, understand as a team
what is important to users, and
consolidate the data
We are only going to review this first
half of the contextual design process.
40. November
2014
Researched
different user
research
methods
Defined the
goal of
contextual
inquiry
Planned the
interview
questions
Prepared IRB
application
December
2014
Applied for IRB
approval
Obtained
money for
participation
February
2015
January
2015
Prepared
materials and
gave training
about
contextual
inquiry to Web
Services dept.
Study was
awarded
Exempt status
by IRB
Refined
interview
questions
Recruited and
trained library
stakeholders
41. March
2015
Ran pilot
interviews
Recruited and
interviewed
users
Conducted
interpretation
sessions
April
2015
Continued to
interview users
Conducted
interpretation
sessions
Created
sequence
models
Created
personas
May
2015
Consolidated
sequence
models
Did affinity
diagramming
Did visioning
Prepared final
report
43. The interview process
Interviews were scheduled in advance at location where
user typically did research
Interviewer and note taker conducted
1-hour interviews with each user
Utilized master/apprentice model
Each participant received a gift card
Captured audio as well as hand-written notes
2-hour interpretation session followed each interview
within 24 hours
49. Undergraduates (8)*
Geology and Geophysics
Civil and Environmental
Engineering
Biochemistry
Business Finance
Graphic Design
Chemical Engineering
History
English Literature
Graduate Students (3)
Construction Management
Social Work
Library & Information
Science
Faculty (1)
Classical Rhetoric
* Three of the undergrads were also students in the Honors College
51. M.D. Anderson Library
Quiet zones
Business zones
Learning Commons
Instruction Room 106-R
Academic Research Center
Department of Earth & Atmospheric Science
Computer Lab
Student Center – Legacy Lounge
The Nook coffee shop
Faculty member’s office
53. Goals
1. To make sense of data
2. To understand intent
3. To move towards insights
54. Interviewer tells the story of the interview
Interview note taker helps tell the story
Interpretation session note taker records
thoughts and observations of the team as notes
for an affinity diagram
Moderator keeps the meeting focused on the
session objectives and makes sure everyone is
involved
Designer generates sequence models as the
interview story is told
Stakeholders contribute thoughts and
observations
61. Interpretations of events, use of tools,
problems, and opportunities
Important characteristics of the work
Breakdowns in the work
Cultural influences
Design ideas
Questions or uncertainty about something from
the interview
Insightful user quotes
62. Thoughts were captured in a Word document
template so we could easily print them out onto
sticky notes.
64. At the end of the interviews and interpretation
sessions, we had:
User profile data that helped us build personas
Captured notes that were combined to build an
affinity diagram
• We generated ~75 notes per interpretation
session
A list of insights from discussing the data
A series of sequence models for an individual
participant
66. A sequence model is a diagram that
show the order in which each
participant completed each of their
tasks.
A consolidated sequence model is a
diagram that displays all the
participants’ diagrams together.
67. A model provides a shared
understanding of the user data, a shared
language for the design team, and an
easily understandable deliverable for
communication outside the design team.
- Jon Kolko, Founder
& Director of The Austin
Center for Design
http://www.jonkolko.com/projectFiles/scad/IDUS215_03_Ethnography_AnalyzingData1_ContextualDesign.pdf
68. The step: The actual thing the user did at the
appropriate level of detail.
The trigger: The situation(s) that prompts a user to
start a new task or a particular step. A trigger always
starts a sequence.
The intent: The reason, known or unconscious, the
user is doing the task or the step. The more intents
you can identify, the better for your future design.
A breakdown: A point in the sequence where the
direct path to fulfilling the intent breaks down, and
the user must devise a workaround or quit.
69. • Interpretations of events, use of artifacts, problems,
and opportunities
• Important characteristics of the work
• Breakdowns in the work
• Cultural influences
• Design ideas (flag with DI:)
• Questions for future interviews (flag with a Q:)
• Insightful customer quotes
https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/contextual-design
70.
71.
72.
73. The sequence models help
direct design decisions
on what to do as well as
what not to do.
74. We learned what breakdowns exist
and can make design decisions
to help users more easily recover
from the inevitable breakdowns.
76. Hierarchical representations of the issues for
your user population built from interpretation
session affinity notes
Group the data into key issues under labels
that reveal users’ needs
Show the common issues, themes, and
scope across all users
Issues and themes translate well into user
requirements and user stories
78. Top Level: Theme
Second Level: Consolidated
user needs, often articulated in
the voice of the user
Third level: Individual user
needs, always in the voice of
the user
Fourth Level: Ideas, insights,
and observations from the user
interview interpretation
sessions
79.
80.
81.
82. Major themes of our affinities
Discoverability of resources
Valuing physical space and their resources
Information overload
Organization and efficiency
Collaboration and exploration
Lack of confidence and trust
Importance of resource medium
(These are from the green sticky notes.)
83. User needs
Full library discovery
More personalized user experience
Get to resources and info quickly
Find library resources library through Google
DRM free resources
Printer-friendly e-resources
Simple, clean design
Point of need assistance
Mobile friendly
85. Personas are fictional characters created to
represent the different user types that might use
a site, brand, or product.
They embody the characteristics, behaviors,
and needs observed through user interviews.
They help project team members and
stakeholders develop a shared understanding
of what the users might need in a variety of
scenarios.
89. Visioning is where the team uses the
consolidated data to drive conversations
about how to improve users' work.
A vision includes the system, its delivery, and
support structures to make the new work
practice successful.
It sets a possible design direction, without
fleshing out every detail.
https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/contextual-design
92. Contextual Inquiry
Interpretation Sessions
Sequence Modeling
Affinity Diagramming
Visioning
Storyboarding
User Environment Design
Persona Development
Interaction and Visual Design
Paper Prototypes and Interviews
Product and System Requirements
Talk with users, capture key
issues, understand as a team
what is important to users, and
consolidate the data
Set a direction based on what you
know about your users
Inspired by https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/contextual-design
Design a system to support all the
research
Iterate designs and systems with
users
93. What were the challenges of
doing contextual inquiry?
94. It’s time-intensive
It’s resource-intensive
Lead investigators need training
Need familiarity with analyzing large
amounts of data
You need incentives for long interviews
You need space for affinity diagramming
95. What were the advantages of
doing contextual inquiry?
96. It produced an incredible amount of in-depth,
rich qualitative data that can be used by
multiple departments within the library.
We better understand our users and their
research process.
We developed robust personas that can be
used for spaces and services throughout the
library, not just the website.
97. We have a final report to share with
colleagues at UH and other libraries about our
methodology and findings.
We have a better understanding of where
breakdowns exist.
We will be able to create a website that is
more responsive to our users’ needs.
99. Beyer, H., & Holtzblatt, K. (1997). Contextual design: defining customer-centered systems. Elsevier.
Dempsey, L. (2014, November). Thinking about Technology . . . Differently. Speech presented at LITA Forum 2014,
Albuquerque, NM.
Holtzblatt, K., & Jones, S. (1993). Contextual inquiry: a participatory technique for system design. Participatory
design: principles and practice. Hillsdale, NJ: Erlbaum.
Holtzblatt, K., Wendell, J. B., & Wood, S. (2004). Rapid contextual design: a how-to guide to key techniques for
user-centered design..
Kolko, J. (2015, February). How to Use Empathy to Create Products People Love. Speech presented at the
Designing for Digital Conference, Austin, TX.
Makri, S., Blandford, A., & Cox, A. L. (2006). Studying Law students’ information seeking behaviour to inform the
design of digital law libraries. Presented at: 10th European Conference on Research and Advanced Technology for
Digital Libraries 2006 (ECDL2006), Alicante, Spain.
Notess, M. (2004). Three looks at users: a comparison of methods for studying digital library use. Information
Research, 9(3), 9-3.
Portigal, Steve. (2013) Interviewing Users: How to Uncover Compelling Insights.
Raven, M. E., & Flanders, A. (1996). Using contextual inquiry to learn about your audiences. ACM SIGDOC Asterisk
Journal of Computer Documentation, 20(1), 1-13.
Rosenfeld, Lou. “Seeing the Elephant: Defragmenting User Research.” http://alistapart.com/article/seeing-the-
elephant-defragmenting-user-research
100. Thank you!
Rachel Vacek
Head of Web Services
University of Houston Libraries
rachelvacek.com
vacekrae@gmail.com
@vacekrae
All our user research is publically available at:
sites.lib.uh.edu/wp/website-redesign/
This presentation is available on:
slideshare..net/vacekrae
Editor's Notes
This presentation, although about a contextual inquiry conducted by a web department, can be applied to ANY type of service – physical spaces and services as well as online systems and services.
The three circles of information architecture
The infamous three circle diagram to be a great tool for explaining how and why we must strike a unique balance on each project between goals and context, user needs and behavior, and content.
The UX Honeycomb
The honeycomb hits the sweet spot by serving several purposes at once. First, it’s a great tool for advancing the conversation beyond usability and for helping people understand the need to define priorities. Is it more important for your web site to be desirable or accessible? How about usable or credible? The truth is, it depends on your unique balance of context, content and users, and the required tradeoffs are better made explicitly than unconsciously.
Ethnography is the study of people in their own environment through the use of methods such as participant observation and face-to-face interviewing.
Human computer interaction engineers at the Digital Equipment Corporation (DEC) as a way to understand digital products from the perspective of their users. It’s evolved, but the protocols in this book have remained to be the best practices for contextual inquiry across both private and public domains.
Contextual inquiry was first referenced as a “phenomenological research method” in a paper by Whiteside, Bennet, and Holtzblatt in 1988, which lays out much of the justification for using qualitative research methods in design. It was first fully described as a method in its own right by Wixon, Holtzblatt, and Knox in 1990, where comparisons with other research methods are offered. It is most fully described by Holtzblatt and Beyer in 1995.
Contextual Inquiry was extended to the full Contextual Design methodology by Beyer and Holtzblatt between 1988 and 1992. Contextual Design was briefly described by them for Communications of the ACM in 1995, and was fully described in Contextual Design in 1997.
Work models as a way of capturing representations of user work during interpretation sessions were first briefly described by Beyer and Holtzblatt in 1993 and then more fully in 1995.
In 2010, the NCSU Libraries worked with a design firm (Hesketh.com and More Better Labs) to conduct a contextual inquiry for their website redesign
Tacit knowledge is knowledge about their own work process that users themselves are not consciously aware of.
Contextual inquiry has the following limitations:
1) As a qualitative research technique, the results from a contextual inquiry may be inadequate for conducting statistical inference. If 50% of users studied raised a specific issue, it cannot be concluded that 50% of the population experiences that issue. Follow-up surveys can provide quantitative sizing information where needed.
2) Contextual inquiry is resource-intensive. It requires travel to the informant's site, a few hours with each user, and then a few more hours to interpret the results of the interview.
Contextual inquiry is really just the first step in contextual design. This means thinking and planning out how things relate to one another, and how they are useful, usable, accessible, findable, desirable, credible, valuable.
Only going to cover this first part of the contextual design process. Getting the data and making sense of it, and figuring out where you need to go.
Jon Kolko, Austin Center for Design, Contextual Design Work Modeling presentation
Rapid Contextual Design,
An Insight is a major work observation or lesson about the users’ work practice and application experience that has significant implications for the design of the system. Insights are captured at the end of each interpretation session.
Rapid Contextual Design,
“A step-by-step recording of the tasks observed or retrospective accounts recorded during the Contextual Interview.”
Kolko, Contextual Design Work Modeling pres, ac4d
“Most importantly, models give us a visual representation of the user data.”
The hierarchy is typically represented on sticky notes with colors that correspond as follows:
Contextual inquiry is really just the first step in contextual design. This means thinking and planning out how things relate to one another, and how they are useful, usable, accessible, findable, desirable, credible, valuable.