Overview

Evaluating Interactive Systems Lecture 5
AT 71.06 : Human Computer Interaction Paul Janecek 16 February 2005
Admin Near future, Project, Homework Evaluation & Critiquing Tools Expert Reviews, Usability Testing
Near Future: Next week: 23 February: Holiday 24 February: Poster presentations Following week: midterm exa! review: chapters 1!5 course notes articles

Projects

Due today: OAI models, HGD models Next week: Scenarios of use for 3 primary tasks "written# Poster presentations "10 minutes max# Explanation of problem, users, tasks State charts "with or without screens# Walkthrough scenarios

Homework

3L: GOMS KLM analysis of mobile phone Show the code, description and time for each action Ideally, we would use the same times for the same
actions in the same conditions Group actions into goals/subgoals

Task 2 (msec)

3L: GOMS KLM analysis of mobile phone

Mobile Phone Comparison

15000 Task 1 (msec) Codacom C400 Motorola T720 Nokia Nokia 2100 Nokia 2300 Nokia 3120 Nokia 6230 Nokia 6510 Nokia 8300 Panasonic Panasonic EBG51M Samsung Samsung X430 Siemens ME45 Sony Ericsson K508i Sony Ericsson T610
20 Codacom C400 Motorola T720 Nokia 2100 Nokia 2300 Nokia 3120 Nokia 6510 Nokia 8300 Panasonic Panasonic EBG51M Samsung X430 Siemens ME45 Sony Ericsson K508i Sony Ericsson T610 30
25 Tasks 1&2 (total steps)

15 Task 2 (steps)

The Winner: Nokia
20000 Tasks 1&2 (total msec)
Codacom C400 Motorola T720 Nokia 2100 Nokia 2300 Nokia 3120 Nokia 6510 Nokia 8300 Panasonic Panasonic EBG51M Samsung X430 Siemens ME45 Sony Ericsson K508i Sony Ericsson T610

Task 1 (steps) 15 20

Evaluation and Critiquing Tools

Expert Reviews

Usability Testing Surveys

Expert Reviews

Heuristic Evaluation Guidelines Review Consistency Inspection Cognitive Walkthrough Formal Usability Inspection
Acceptance Tests Evaluation During Active Use Controlled Experiments
Report Comprehensive rather than opportunistic Based on Guidelines Novice!Expert, Consistency Based on Theory OAI: evaluation of tasks and actions Rank by importance and expected e%or
Procedure Environment similar to users Bird$s!eye view detecting patterns/inconsistencies Software tools search for consistency in docs web!accessibility validation
Problems General experts may not have adequate
understanding of domain or user community

Usability Testing

Accepted by industry "e.g., Google# Academics vs. marketing Controlled experiments to test hypothesis result: validation/rejection of hypothesis Market studies to test sell!ability result: recommendations for changes
Domain experts may have di&culty

imagining novice

Description rooms separated by mirrored window sta% with expertise in UI design participates in early task analysis and

design meetings

Test Plan 2!6 weeks in advance list of tasks questionnaire for subjective satisfaction Pilot test 1!3 participants, 1 week ahead procedures, tasks, questionnaire

helps develop tasks for testing
Test Participants represent intended user community attention to background info to avoid bias 'not they who are being tested; rather it is
the interface that is under study(
Think!aloud protocols often reveals how user$s perceive system designer or tester should be supportive of

participants

informal atmosphere encourages

participation

two participants even better
Video valuable for review convincing evidence negative: time!consuming to review Software logging ping, mouse events, screens, timestamp

Variations

1. Paper mockups administrator plays role of system 2. Discount usability 3!6 participants advocates: serious )aws caught quickly critics: complex systems require complete coverage compromise: do both
Variations "continued#
3. Competitive: compare systems 4. Universal: test in diverse con*gurations
Expert Reviews Usability Testing Surveys Acceptance Tests Evaluation During Active Use Controlled Experiments
international, small/large displays
5. Field tests 6. Remote Usability 7. Can!you!break!this test

Continued Tomorrow