FinalReport
Group 1: Samantha Dietz, Sarah Dulski, Gosia Dzienisowicz, and Mary Remington HCI460 Usability Evaluation Methods 10/28/2009

TableofContents

ExecutiveSummary___________________________________________________________________2 Introduction_________________________________________________________________________3 Methodology________________________________________________________________________3 Finding&Recommendations____________________________________________________________5 HighSeverityIssues_____________________________________________________________7 MediumSeverityIssues_________________________________________________________11 LowSeverityIssues____________________________________________________________14 PositiveFindings______________________________________________________________15 Appendix___________________________________________________________________________16 1|P a g e

ExecutiveSummary

ThemainobjectiveofthefollowingstudywastoevaluatetheSamsungSyncMasterP2370HDremote controlandtelevisiondisplayintermsoftheuserexperiencethatitcreates.The5participants,3male and2female,ages26to55,wereaskedtocompletefourtasksinvolvingtheremotecontroland televisionmenu.Thesetasksincludedprogrammingtheinitialchannellist,changingachannel,changing thevolume,andaddingafavoritechannel.AlltestsessionswereconductedonOctober18,2009inthe livingroomofanapartmentbelongingtooneoftheresearchers. Oncethedatawasanalyzed,threemainoverarchingrecommendationsbasedofthefindingswere found;simplifycommontasks,fulfilluserexpectations,andintuitiveinteractionbetweentheremote controlandtelevisiondisplay. SimplifyCommonTasksTasksdonerepeatedlyshouldbedesignedsothattheyareeasyfor theusertocompletewithminimalguidance.Unnecessarystepsshouldbeeliminated;andif possible,thetaskshouldbeturnedintoaonestepprocess.Doingthiswillincreaseeaseofuse andincreasetheuserssatisfactionwiththeproduct. FulfillUserExpectationsAsaremotecontrolisadevicethatpeoplemayuseonadailybasis, theyarefamiliarwithitspurposeandhavecometoexpectspecificinteractions.Thisisan opportunitytoleveragethoseuniversalinteractionstocreateabetteruserexperiencewiththe SamsungSyncMasterP2370HD. IntuitiveInteractionbetweentheRemoteControlandTelevisionDisplayByimplementing bettervisualcuesonboththeremotecontrolandtelevisiondisplay,theuserwillbetter understandhowtogetthroughcomplextasks. 2|P a g e

Introduction

Productevaluated OurtestingsoughttoexaminetheusabilityoftheSamsungSyncMasterP2370HDtelevisionremote controlandtelevisiondisplay.Itisa23inchflatscreenLCDtelevisionthatisbundledwithafully functioningremotecontrol. Objectivesofthestudy TheobjectiveofthestudywastoevaluatetheSamsungSyncMasterP2370HDremotecontroland televisiondisplayintermsoftheuserexperiencethatitcreates.Althoughmanyfunctionscanbe performedviathetelevisionconsole,theremoteistheprimarytoolusedinconjunctionwiththe televisiontonavigatethroughthechannelsandthemenuonthetelevisionscreen.Becauseofthis,the remotecontrolneedstobeintuitivetouseoreasilylearnable.Also,thetelevisionscreenmenumust operateinamannerthattheuserswouldexpect. WetargetedpotentialSamsungcustomersandaskedthemtoperformtasksonthetelevisionwiththe remotewhilewerecordedtheiractions.Wetestedwhetherthecurrentremotecontroldesignand televisionmenulayoutwereconducivetoperformingcommontaskssuchaschangingthechannel, programmingthechannels,andalteringthevolume. Specificallyweansweredthefollowingquestions 1. Dousersunderstandhowtocompleteprimaryfunctionality?Primaryfunctionalityincludes,but isnotlimitedto,poweringon/off,channelchanging,andvolumeadjustments. 2. Dousersunderstandhowtocompletesecondaryfunctionality?Secondaryfunctionality includes,butisnotlimitedto,programmingthetelevisionchannelsandsettingfavorite channels. 3. Areusersabletocompleteprimaryandsecondarytaskswithoutreadingthehelp documentation? 4. Iftheuserrequiresadditionalinstructions,isthehelpdocumentationwrittenclearlyenoughfor theusertounderstandandcompletethetaskinthefirsttrywithoutfrustration?

Methodology

TestingLocation TheusabilitytestingwasconductedbytheProtochicusertestingteam.Severalpilottestswererun,and theofficialtestingtookplaceonOctober18th,2009.Itwasdoneat1447W.BalmoralAve.Apt.3N Chicago,IL60640inthereallivingroomofoneoftheProtochicstaffmembers. Procedure TheparticipantssatonacouchfacingacoffeetableandtheSamsungtelevision.Onthecoffeetable wasthetelevisionremotecontrol.Themoderatorsatonachairnexttothecouchsothatthe participantsactionscouldbeobservedwithlittleinterference. Cameraswereinpositiontorecordtheinteractionoftheusershandontheremote,theusersfacial expressions,andtheinteractionsonthetelevisionscreen.Inaddition,twootherProtochicmembers wereintheroomtoobserveandtakenotesonthetestsessions. 3|P a g e
Duringeachusabilitytest,themoderatorstartedbywelcomingtheparticipant,introducingthe Protochicteam,andcollectingthesignedconsentforms.Additionally,themoderatordescribedwhat objectwasgoingtobetestedandhowthetestsessionwasgoingtowork. Themoderatorthenaskedtheparticipantfivewarmupquestionsregardingtheirtelevisionwatching habits: 1. Howoftendidshe/hewatchTV? 2. Whatkindsofshowsdidshe/helike? 3. WhatbrandofTVdoesshe/heusethemost? 4. Howoldisher/hiscurrentTV? 5. DoestheparticipantprefertousetheremotecontrolorthebuttonsontheTVtochangethe channels? Oncethewarmupquestionsconfirmedthattheparticipantdidfitthecriteriaforthestudy,the moderatorthenverballyexplainedfourdifferenttasksfortheparticipanttoperformoneatatime.The moderatoralsoplacedawrittencopyofthetaskinstructionsonthecoffeetabletohelpreducethe participantsmemoryload. Theparticipantswereaskedtocompletethefollowingfourtasks: 1. Programtheinitialchannels 2. Changethechannel 3. Adjustthevolume 4. Programafavoritechannel Theparticipantswereaskedtoverbalizetheirthoughtprocesswhiletheyworkedthroughthetasks. Aftereachtaskwasperformed,themoderatoraskedtheparticipantstoreflectontheirexperiencesand answerquestionsregardingthecompletedtask.Theparticipantswerequestionedtoseewhetherthey thoughtthetaskwaseasyordifficulttocompleteandwhytheythoughtso.Theywerealsoaskedifthis wasduetoanydesignaspectoftheTVdisplayand/orremotecontrol. Aftertheparticipantshadcompletedallfourtasks,themoderatoraskedthemabouttheiroverall experienceusingtheremotecontrolandtelevisionmenu.Themoderatoralsoaskediftheparticipants wouldpossiblybuytheSamsungtelevisionbasedontheiruserexperience. Attheendofthesessionthemoderatorthankedtheparticipantsfortheirtimeandvaluableinsights. Oncethelastparticipanthadleftthetestingsite,theProtochicteammettodiscusstheirfindings. Participants Atotalof5participantspassedthescreenerandfitthecriteriafortheSamsungtelevisiontesting.They wereallchosenfromacquaintancesoftheProtochicteam.Theparticipantsrangedfromage26to55, includingthreemaleandtwofemalewithvariouslevelofexpertiseandTVwatchinghabits.Someof theparticipantswatchTVdailywhileotherswatchthreetofivetimesaweek.Theyalllikeavarietyof showslikecomedies,dramas,anddocumentaries.NoneofthemownedaSamsungTVandtheyalluse aremotecontroltooperatetheirtelevisions.Itisnoteworthythattwooftheparticipantshadphysical disabilities.Participant1washearingimpairedandParticipant2wascolorblind.PleaseseeAppendix1 fordetails. 4|P a g e

Findings&Recommendations
Overview Inthissectionofthereportarethefindingsconcludedfromthedataandrecommendationsonhowto solveusabilityissues. SuccessorFailureofTask Foreachtask,wemeasuredthesuccess/failurerateofeachparticipant.Ataskwasconsideredasuccess iftheparticipantcompletedthetaskwithoutmajorproblemsorhelpfromthemoderatororthe instructionmanual.Ataskwasconsideredafailureiftheparticipantdidnotcompletethetaskorwas onlyabletocompletethetaskwithhelpfromthemoderatororinstructionmanual. AlloftheparticipantssuccessfullycompletedTasks2,3,and4(changingthechannel,adjustingthe volume,andaddingafavoritechannel).DuringTask1(programmingtheinitialchannel)oneoutoffive participantsfailedtocompletethattask.PleaseseeAppendix2fordetails. ExperienceandEaseofUse Attheendofeachtask,theparticipantwasaskedtoratehowexperiencedhe/shewasinperforming thattask(scalefrom1veryexperiencedto3inexperienced)andtoratetheeaseofuseofthedevicefor thatparticulartask(scalefrom1veryeasyto4verydifficult). DuringTask1(programmingtheinitialchannels)onaveragetheparticipantsweresomewhat experiencedwiththistypeoftaskandrateditassomewhatdifficulttocomplete. DuringTasks2and3(changingthechannel,andadjustingthevolume)alloftheparticipantswerevery experiencedwiththesetypesoftasksandratedthemasveryeasytocomplete. DuringTask4(addingafavoritechannel)onaveragetheparticipantsweresomewhatexperiencedwith thistypeoftaskandratedthemasbeingbetweensomewhateasytosomewhatdifficult.Pleasesee Appendix2fordetails. Attheendofthetest,participantswereaskedtoratetheoveralleaseofuseofthedevice.Onaverage, theparticipantsratedtheSamsungtelevisionassomewhateasy.PleaseseeAppendix2fordetails. Attheendofthetest,eachparticipantwasaskedhowlikelyitwasthattheywouldpurchasethis televisionifmoneywasnotafactor(scalefrom1Iwouldbuyittodayto4Iwoulddefinitelynotbuyit). Theanswersamongthefiveparticipantsvariedfrom1to5,medianbeing2(Iwouldthinkaboutbuying itinthenearfuture). Similarly,theparticipantswereaskediftheywouldbeinterestedinpurchasingthistelevisionifalltheir suggestionsforimprovementswereimplementedscalefrom1Iwouldbuyittodayto4Iwould definitelynotbuyit).Theanswersamongtheparticipantsvariedfrom1to4,againwiththemedian being2(Iwouldthinkaboutbuyingitinthenearfuture).PleaseseeAppendix2fordetails. 5|P a g e

IncludedareanumberofissuesandrecommendationsfortheSamsungtelevisionandremotecontrol. Eachissueisaccompaniedbyaseverityrating.Theseratingsarebasedonthefrequencywithwhichthe problemoccurred,theimpactoftheproblemontheuserinteraction,thepersistenceoftheproblem. Isitaonetimeproblemthatuserscanovercomeoncetheyknowaboutitorwillusersrepeatedlybe botheredbytheproblem? Therearefourcategoriesofseverityratings: HighAseriousconditionthatimpairstheoperation,orcontinuedoperation,ofoneormore productfunctionsandcannotbeeasilycircumventedoravoided.Thedevicedoesnotprevent theuserfrommakingaseriousmistake.Theusabilityproblemisfrequent,persistent,and affectsmanyusers. MediumAnoncritical,limitedproblem.Itdoesnothinderoperationandcanbetemporarily circumventedoravoided.Theproblemcausesusersmoderateconfusionorirritation. LowNoncriticalproblemsorgeneralquestionsabouttheproduct.Thereareminor inconsistenciesthatcauseaslighthesitationorsmallaestheticissues. PositiveAspectsoftheinterfacethatallowtheusertocompletefunctionalitywiththe interfacewithease. 6|P a g e

HighSeverityIssues

Finding1[Help/InfoBox]
Problem Althoughthereisahelp/infotextboxatthebottomofthescreenwithadescriptionoftheselecteditem onthemenu,noneoftheparticipantsnoticedit.Withoutthiskeyinformation,participantswere unsureofhowtocompletethetaskwithinthetelevisionmenu.Thelocationofthehelp/infoboxatthe bottomofthedisplayawayfromtheMainMenu(taskarea)doesnotencouragetheusertoassociate thetwotogether. SupportingEvidence DuringTask1and4(programmingtheinitialchannelsandaddingafavoritechannel)noneof the5participantsverballymentionedthattheysawtheoptionsintheinformationbox. Threeoutoffiveparticipantsstatedthattheywereunsurehowtocompletethesetasksand theywereguessingwhichbuttonstheyshouldpress. Oneparticipantnoticedtheadditionalhelptextatthebottom,butdidnotthinkitwas informativeenoughtoexplainthescreenoptions. Recommendation Betterintegrationofthehelp/infoboxwiththemainmenu(taskarea)sothattheuserwillassociatethe buttonsthattheyarepressingandtheinformationassociatedwiththem.Forexamplemovethe informativetextdirectlytotherightofthemenucurrentlyinfocus. Thiswillmakethehelp/informationmoreimmediatelyvisibleandhelpguideorinformtheuserifthey areuncertainofwhattodo.
Figure1:AutoProgramScreen

7|P a g e

Finding2[ToolsButton]
Problem TheparticipantsdidnotrecognizethattheToolsbuttonneededtobeusedinordertoaccesssomeof thetelevisionfunctionality. SupportingEvidence EventhoughtheToolbuttonwasintegraltocompleteTask4(addingafavoritechannel),allof theparticipantstriedtocompletethetaskbystartingwiththemenubutton.Itwasonlyby accidentorguessingthattheusersrecognizedtheToolButtonontheremoteaswellasthe callouttoitsfunctionalityonthemenudisplayinthehelp/infobox. DuringTask4(addingafavoritechannel)3outof5participantswerelostandshowed frustrationuntiltheyaccidentallypressedtheToolsbutton. Oneparticipantstated,Goingtotoolswasnotmyintuitivechoice.didntrealizeitwouldhelp me. Recommendation AstheTooldisplayfunctionalityisasubsetoftheMenudisplayfunctionality,itisrecommendedthat thetwobuttonsvisuallythroughtheirshape,colorandplacementontheremoteshowthesame relationship.

Figure2:ChannelListscreen

8|P a g e

Finding3[Ch.ListButton]
Problem Ch.Listdidnotmeetparticipantexpectationsofhowitshouldfunctionthereforecausingparticipant frustration. SupportingEvidence Duringtask1(programmingtheinitialchannels)3outof5participantsexpectedthatby pressingtheCh.ListbuttoneitherthetelevisionwouldautomaticallyAutoProgramoramenu wouldappeartohelpthemcompletethetask. WhenpressedtheCh.Listbuttontakestheusertoamenuwiththechannelsthatare currentlyinthechannellist.Thisisasubmenuofthemainchannelmenu. Recommendation TheusershouldbeabletocompleteallchannelrelatedtaskswhenclickingontheCh.Listbuttonas thiswasfunctionalityexpectedbytheparticipants.Thesetasksinclude,butarenotlimitedto, adding/deletingachanneltothechannellistandaddingallinitialchannelstothechannellist.

Finding4[Fav.ChButton]

Problem ParticipantsexpectedthatbypressingandholdingtheFav.Chbuttonthetelevisionwould automaticallysavethecurrentchanneltofavorites.TheFav.Chbuttonontheremotecontroldidnot followuserexpectationsthereforemakingitmoredifficultforthemtocompleteprogramming functionalityandcreatingfrustrationwiththeinterface. SupportingEvidence TheFav.Chbuttonisnotcapableofdoingtheexpecteduserflowcreatingfrustrationwiththe interface.InthecurrentstatewhenafirsttimeuserclickstheFav.Ch,anerrorscreenstating NoFavoriteChannelsaresavedappears. DuringTask4(addingafavoritechannel)3outof5participantsthoughtthattheycouldhold downtheFav.Chbuttontoaddafavoritechannelbasedofftheirexperienceswith competitorsremotes. Recommendation AddsmartbuttonfunctionalitytotheFav.Chbuttonsothatwhenitishelddownthecurrent channelwillbeaddedtothefavoritechannellist. 9|P a g e

ImageforFinding3&4

Figure3:Remotecontrol

10|P a g e

MediumSeverityIssues
Finding5[ArrowKeyIconography]
Problem ArrowKeyiconographyistoosmalltorecognizeatanappropriateviewingdistancefromthetelevision. Participantsimmediatelyunderstoodup/downfunctionalityoftheremotebutwereconfusedonhowto navigatetothesubmenus. SupportingEvidence Allparticipantsdidnotvocalizerecognizingtherightarrowicononthedisplayandcorrelatethe functionoftherightarrowkeybuttonontheremoteasawaytonavigatebetweentheMain MenuandSubMenus. DuringTask1(programminginitialchannels)2outof5participantsdidnoticetherightarrowon themainnavigationiconsbutdidnotrecognizeitasanavigationtoolandcontinuedtobe confusedonhowtomovetothesubnavigationmenu.Ithoughtitwasabulletpoint. Recommendation Maketherightarrowonthemainnavigationiconatleasttwiceaslargetohelpindicatethattheuser mustpresstherightarrowbuttonontheremotetoaccessthesubnavigationontheTVdisplay.
Figure4:AutoProgramScreen

11|P a g e

Finding6[ColoredButtons]
Problem Whentheparticipantsusedtheremote,evenwhentheywerehavingdifficultywithataskandexploring buttonfunctionality,noneofthemusedthefourcoloredbuttonswithoutlabelsatthebottomofthe remote.Astheyarenotlabeled,participantsdidnotthinkthattheywereimportanttothetaskanddid notexplorethem. SupportingEvidence Thecoloredbuttonsontheremotewerelabeledonthetelevisionscreenmenu.Userswere unabletoseethisconnection,andwereunabletousethebuttonstocompletetasks.Thefour coloredbuttonslocatedatthebottomoftheremotecontrolarerequiredtocompletesomeof thetaskswithinthetelevisionmenuandhavedescribedfunctionalitywithinthehelp/infobox. However,theparticipantswerenotabletocorrelatethecolorediconsonthetelevisiondisplay withthesamecoloredbuttonsontheremotecontrol.Therefore,theparticipantsdidnotuse themtocompleteanytask. DuringTask4(addingafavoritechannel)4outof5participantsdidnotrecognizetheonscreen labelingofthecoloredbuttonswhichcausedthemtonotusethebuttonsforthetask. Participant2:Isawthecoloredoptionbuttons,butdidn'tthinktousethem.Didn'tmake sensetomeformetousethem." Recommendation Ifthefourcoloredbuttonsontheremotecontrolonlyprovideonefunction,theyshouldbelabeled accordingly.Forthetaskswetestedthegreenbuttonwasusedtozoominandoutofthetelevision displaythereforeitshouldbelabeledZoom.Theyellowbuttonwasusedtoselectchannelstoaddto thefavoritechannellistthereforeitshouldbelabeledSelect.

Figure5:ChannelListScreen(underMenu)

12|P a g e

Finding7[TerminologyontheMenuDisplay]
Problem UsersdidnotunderstandtheterminologyusedintheAutoProgrammingMenuDisplay.Thelanguage wastootechnicalforauserwithoutanaudiovisualbackgroundtounderstand. SupportingEvidence Differentscreenswithinthemenudisplayincludedterminologyoracronymsthatwerenot recognizable.Astheparticipantsdidnotunderstandtheterminology,ithamperedtheirability tocompletethetaskswithouthesitationorfeelingasiftheyguessedthroughthetask. DuringTask1(programminginitialchannels)3outof5participantswereunsureHRCand IRCmeant. Recommendation Uselanguagethatthemajorityofuserswillunderstandsincenotalloftheusershavethesamelevelof comprehensiontotermsthatareaudiovisualspecific.Additionally,includethemeaningandthe definitionsforthetechnicallanguageandacronyms.
Figure6:AutoProgramScreen
LowSeverityIssues Finding8[PowerON/OFFButton]

13|P a g e

Problem Televisionturningonandpoweringupdoesnotmeetparticipantsexpectations.Whenparticipants pressedthepowerbuttontheirexpectationswerethatthedisplayscreenwouldindicatethetelevision waspowering"on"anditwouldhaveamoreimmediateresponsetime. SupportingEvidence Twouserspressedthepowerbuttonmorethanonce,thinkingthattheremotehadnot registeredtheirfirstclick.Thesesecondaryclicksthencausedthetelevisiontoimmediatelyturn off. Participants3explicitlystatedhisfrustrationwiththeslowresponsetimefromthetelevision whenhepressedthepowerbutton. Thetelevisiontakesthreesecondstopoweron. Recommendation Theremoteneedstoindicatetotheparticipantthatithasregisteredtheirbuttonclickeitherbyalight ontheremoteoramessageonthedisplay.Thetelevisiondisplayneedsonstartuptoshowanactive (activemeaningpicture,menuorsnow)display.Thecolorofthelightsneedstobemoreprominentto beviewedbytheparticipant.
Figure7:TelevisionPoweringUp

LowSeverityIssues

Finding9[ButtonsFunctionalityProximity]

Problem 14|P a g e

Participantsexpectedtofindcertainfunctionalitysuchasclosecaptioningincloserproximitytomute andvolumebuttons. SupportingEvidence Participant1stated"IlikethattheMutebuttonisbyVolume.AlwaysfeltthatCC,Volume ButtonsandMuteButtonshouldbegroupedtogether" Recommendation BecauseCCisrelatedtovolumeasthevisualcounterpartofsound,itwouldbebeneficialtomovethe locationoftheCCbuttonincloserproximitytovolumecontrols.

PositiveFindings

Finding10 Theparticipantswereabletocompletetheprimaryfunctionalitysuchaschangingthechannelsand adjustingthevolumeveryeasily. Finding11 Theparticipantsweresatisfiedwiththeergonomicsizeandshapeoftheremotecontrolthatallowed themtoeasilymaneuverbetweenbuttonswithonehand. Finding12 Allparticipantsagreedthatthemutebuttonwasplacedinaprominentpositionnexttothevolumeand wasclearlylabeled,makingiteasytofind.Additionally,theyalllikedthemuteiconfeedbackontheTV displayafterpressingdownthemutebutton.

Appendix

Appendix1ParticipantDetails

15|P a g e

Gender Male Female TOTAL

Age 4655 TOTAL

TelevisionUsage Everyday Afewtimesaweek Onceaweek Lessthanonceaweek TOTAL AgeoftheParticipantsTV Lessthanayear Ayear 23years 4+years TOTAL
ParticipantsBrandofTV Panasonic 2 Sony 1 Hitachi 1 Elements 1 TOTAL 5 UsesRemotetoChangeChannels Yes 5 No TOTAL
Appendix2TaskRatings Successrateforeachtask(Pass/Fail) Task1 Task2 Task3 Task4 P1 Fail Pass Pass Pass P2 Pass Pass Pass Pass P3 Pass Pass Pass Pass P4 Pass Pass Pass Pass P5 Pass Pass Pass Pass Experiencerateforeachtask(Scale13) 1=VeryExperienced 2=SomewhatExperienced 3=Inexperienced 16|P a g e
Task1 Task2 Task3 Task4 P3 P1 P2 P2 P3 AVG: 2.1 2.2 Overalleaseofuseforeachtask(Scale14) 1=VeryEasy 2=SomewhatEasy 3=SomewhatDifficult 4=VeryDifficult PPPPPAVG: 1.8 Easeofuseforeachtask(Scale14) 1=VeryEasy 2=SomewhatEasy 3=SomewhatDifficult 4=VeryDifficult Task1 Task2 Task3 Task4 P1 P4 P1 P3 P3 AVG: 1.2 2.4 Howlikelywouldtheparticipantspurchasethistelevisionifmoneywasnotafactor(Scale14)? 1=Iwouldbuyittoday 2=Iwouldthinkaboutbuyingitinthenearfuture 3=Iprobablywouldnotbuyit 17|P a g e
4=Idefinitelywouldnotbuyit PPPPPAVG: 2.4 Howlikelyisitthattheparticipantswouldpurchasethistelevisionifalltheirsuggestionswere implementedandmoneywasnotafactor(Scale14)? 1=Iwouldbuyitassoonasthenewversioncameout 2=Iwouldbeinterestedinpurchasingitsoonafteritcameout 3=Iprobablywouldnotbuyit 4=Idefinitelywouldnotbuyit PPPPPAVG: 2.2

18|P a g e

ModeratorsGuide

Group 1: Samantha Dietz, Sarah Dulski, Gosia Dzienisowicz, and Mary Remington HCI460 Usability Evaluation Methods 10/28/2009
PreparingfortheTestSessions
Atthestartofthetestingday Makesurethetelevisionissetupproperlyandintheoffstate. Setuptheallvideocameras(withaudio)torecord: o Theparticipantsfaceandupperbody. o Theparticipantsactionsontheremotecontrol. o Thetelevisiondisplay. Printenoughconsentformsforthenumberofparticipantsforthatday. Eliminateasmanydistractionsaspossiblefromthetestingarea. Printcardswithinstructionsforeachtask(printonecardforeachtask). Beforeeachsession Clearthechannellistonthetelevision. Clearthefavoritechannellistonthetelevision. TurntheTVoffandplacetheremoteonthecoffeetable. Provideaconsentformtotheparticipant. Aftereachsession Noteissuesandsurprises. Debriefwiththeusertestingteam. ResearchTeamResponsibilities Moderator: o Guidestheparticipantthroughtheentiretestingsession. o Answerparticipantsquestions. o Takenotesifdeemednecessary(notrequired). VideoRecorder: o Setsuptelevisionandvideorecordingequipmentandensurestheyallwork properly. o Recordseachtestingsession. Notetaker2: o Recordsanswersforposttaskandposttestquestions. o Takenotesonthebehaviorsoftheparticipantsaswellastheircomments. Notetaker3: o Recordsanswersforposttaskandposttestquestions. o Takesnotesonactionsparticipantcompletesbasedonthebuttons. 1|P a g e

SessionIntroduction

NotetoModerator:Alltextingrayisunspokenandaredirectionstohelpyouthroughthe session. Welcomethetestparticipant,shakehis/herhand,andaskhim/hertopleasebeseatedinthe designatedlocation. Hi,Iam[Name].Whatisyourname? Beforewebegin,mayIhaveyoursignedconsentform? Thankyou. Iftheyhaventbroughtasignedconsentformwiththem,giveittothemnowandhavethem reviewandsign. Doyouhaveanyquestionsabouttheconsentform? Iwillbethemoderatorforyourtestsession.ThismeansIwillbeworkingoffmyscript togiveyoutaskstoperformonthisSamsungremotecontrol. Iwillbeusingthescripttoensurethatyouandtheotherparticipantsaretestedinthe samemanner. Myteammember[Name]willbeinthisroomwithusrecordingthesession.Myother teammembers[Names]willbetakingnotesfromtheotherchairsintheroom.Imay alsotakenotesoccasionallyduringthesession. Asweareindependentresearchers,pleasekeepinmindthatwedidnotcreatethis product.Wearenottestingyouandthereisnosuchthingasawronganswer. Pleasetrytothinkoutloudwhileyoureworkingandbesuretohonestlytelluswhat youthinkaswewillnotbeoffendedbyanyofyourcomments.Telluswhateverisgoing throughyourmindasitcomestoyou.Doingthathelpsustounderstandwhatworksor doesntworkabouttheproduct. Theentiresessionshouldtakeabout4560minutestocomplete. BeforethesessionbeginsIwouldliketoaskyoujustafewquestionsaboutthe televisionyouwatchmost. 2|P a g e

WarmUpQuestions

Askallquestionsasopenendedquestions. 1. HowoftendoyouwatchTV? a. Everyday b. Afewtimesaweek c. Onceaweek d. Lessthanonceaweek[Terminate] 2. WhattypeofTVshowsdoyouliketowatch? 3. WhatbrandofTVdoyouusemostoften? a. Anybrand,withtheexceptionofSamsung(ContinuetoQuestion5) b. Samsung[Terminate] c. Idon'tknow(ContinuetoQuestion4) 4. IsitaSamsung? a. Yes[Terminate] b. No c. Idon'tknow 5. HowoldistheTVyouwatchmostoften? 6. DoyouprefertousetheremotecontrolorthebuttonsontheTVtochangethe channels? a. Remotecontrol b. ButtonsontheTV[Terminate] Inthecaseof[Terminate]: Weappreciateyoutakingthetimetocometoourofficeandparticipateinourstudy. Howeverbasedontheanswertothepreviousquestionsyoudon'tmeetthespecific requirementsthatwearelookingfor.Soatthistimeyouarefreetogo. Pleaseenjoytheappetizersandrefreshmentsandenjoytherestofyourday. Againthankyouforyourtime. Iftheparticipantisnotterminated: Spendminuteexchangingpleasantriesandoffertheparticipantacupofcoffee,sodaorwater. Beforewebegin,wouldyoulikeanythingtoeatordrink? Ourobjectivetodayistoseeyouperformataskasyouwouldnormallydoinyourown home. 3|P a g e

Hereishowthesessionwillwork: Onthetableinfrontofyouistheremotecontrolyouwilluseforalltasks. TherearefourtasksthatIwillaskyoutocomplete. Youllbeaskedtodoonetaskatatime. AtthestartofthesessionIwillgivethesignalto[Name]tostartrecording. OnceIhavesaidPleasebeginyoumaypickuptheremoteandstartthetask. Donotworryifyouhaveanytroubleasyouprogressthroughthetasks.Feelfreeto explorethebuttonsontheremoteandthetelevisionmenuinrelationtothetask. Whenyoufeelyouhavecompletedthetask,sayaloud,ImdonethenturnofftheTV andplacetheremotebackonthetable. Also,pleasekeeptheremoteinthedesignatedsquareboxonthetableatalltimes.This waywecaneasilycaptureyouractionsduringthevideorecording. Aftereachtask,Iwillaskyouacoupleofquestionsregardingthattask. Aftercompletingthefinaltask,Iwillaskyouacoupleofquestionsabouttheoverall experience. AttheendofthesessionIwillgivethesignalto[Name]tostoprecording. Doyouhaveanyquestionsbeforewebegin? Signalto[Name]tostartthesession

4|P a g e

ParticipantTasks

SetUpScenario ImagineyoujustpurchasedanewSamsungflatscreenTV.Youvetakenitoutoftheboxandset itupontheentertainmentcenter. Task1ProgrammingtheInitialChannels TaskfortheParticipant NotestoModerator YourereallyexcitedaboutyournewTV. Thewordbuttonisusedtoreferencepressing abuttonontheremote.Thewordselectis Youfinallygotcablehookedupatyour usedtoreferenceselectinganoptiononthe placeforthefirsttimeandcantwaitto watchallthosegreatShowtimeandHBO menuontheTVscreen. shows.NowyouneedtoprogramyourTV toviewallthechannelsthatareavailable Televisioniscurrentlyintheoffstate. throughyourtelevisionscableprovider. Participantshouldusetheremotetoturnthe Fortestingpurposes,wewillonlyhave TVon,thenpresstheMenubuttononthe youinitiatetheprogrammingprocess. remotetoaccessthemenuontheTV,select Whenyoufeelyouhavecompletedthe Channel,selectautoprogramming,and taskpleaseturnofftheTV. choosethecableantennasourceandpress Pleasebegin. start. Participantstartsthetask. Participantperformsthetaskandthen indicatesthattheyaredonebyturningoffthe TVandplacingtheremotebackonthecoffee table. ObjectivesoftheTask SuccessCriteria Completethetaskwithoutaskingfor Theparticipantwasabletocompletethe help.Thisshowsthattheremotecontrol task. buttonsandTVmenuwereclearenough Theparticipantdidnotaskforany fortheusertounderstand. assistance. Discoverifthebuttonsandthemenuon theTVdisplayareintuitiveenoughforthe participanttounderstandhowtouse them. Observeiftheremote'sdesignwas intuitiveenoughtoknowtostartthistask bypressingthemenubuttonfirst. 5|P a g e

Task1ProgrammingtheInitialChannels WhentoAdministerHelp IssuestoExplore 1. Wastheparticipantabletocompletethe Afterparticipantasksforhelponce,ask taskwithoutaskingthemoderatorfor themtocontinuetoexploretheremote help? andTVdisplay. Iftheparticipantasksforhelpasecond time,askwheretheyarehaving difficultiesthenshowthenextstepin 2. Whatwasthefirstbuttontheuser thepathfromwheretheyare. pressedontheremotecontrol? Iftheparticipantasksfortheinstruction manual,giveittothemtouseforthis task. 3. Atwhichspecificpointofthepathdidthe usershowfrustration? 4. Howlongdidittaketheusertoshow frustration? 6|P a g e
Task1ProgrammingtheInitialChannels PostTaskQuestions 1. Onascaleof13howexperiencedareyouwiththistypeoftask? 1=Veryexperienced 2=Somewhatexperienced 3=Inexperienced 2. Onascaleof14pleaseindicatehoweasyordifficultthistaskwastocomplete. Circleonebasedonparticipantsansweranduseto 1=Veryeasy fillintheblanksforthefollowingquestions 2=Somewhateasy 3=Somewhatdifficult 4=Verydifficult

Easy/Difficult

3. Whydoyouthinkthistaskwas____________tocomplete? 4. Whataspect/softhebuttonsontheremotemadethistask____________tocomplete? a. Placement b. Color c. Labels d. Size e. Other______________________ 5. Whataspect/softheTVdisplaymadethistask____________tocomplete? a. Placement b. Color c. Labels d. Size e. Other______________________ 6. Isthereanythingyouwouldimprove? 7|P a g e
Task2ChangingtheChannel TaskfortheParticipant Nowthatyourchannelsareproperlyset up,youwanttoseewhatiscurrentlyon channeleleven. Notethatforthepurposeofthistaskyou willnotseeapicture,youwillonlysee staticonthescreen. Whenyoufeelyouhavecompletedthe taskpleaseturnofftheTV. Pleasebegin. Participantstartsthetask. Participantperformsthetaskandthen indicatesthattheyaredonebyturningoffthe TVandplacingtheremotebackonthecoffee table. ObjectivesoftheTask Completethetaskwithoutaskingfor help. Discoverifparticipantknewwhich buttonstopresstocompletethetask. IssuestoExplore 1. Wastheparticipantabletocompletethe taskwithoutaskingmoderatorforhelp? 2. Whichmethoddidtheparticipantuse? a. NumberPad b. ChannelUp/Down 3. Didtheparticipanthaveapreferred method?(Thepreferredmethodis relevanttodesignplacementonthe remotecontrol). 4. Howlongdidittaketheusertoshow frustrationifany?
NotestoModerator Observetoseeiftheparticipantjumps betweenthechannelsbyusingthenumber padorbyusingtheup/downarrows.
SuccessCriteria Theparticipantwasabletocompletethe task. Theparticipantdidnotaskforany assistance. WhentoAdministerHelp Afterparticipantasksforhelponce,ask themtocontinuetoexploretheremote andTVdisplay. Iftheparticipantasksforhelpasecond time,askwheretheyarehaving difficultiesthenshowthenextstepin thepathfromwheretheyare. Iftheparticipantasksfortheinstruction manual,giveittothemtouseforthis task.

8|P a g e

Task2ChangingtheChannel PostTaskQuestions 1. Onascaleof13howexperiencedareyouwiththistypeoftask? 1=Veryexperienced 2=Somewhatexperienced 3=Inexperienced 2. Onascaleof14pleaseindicatehoweasyordifficultthistaskwastocomplete. Circleonebasedonparticipantsansweranduseto 1=Veryeasy fillintheblanksforthefollowingquestions 2=Somewhateasy 3=Somewhatdifficult 4=Verydifficult
3. Whydoyouthinkthistaskwas____________tocomplete? 4. Whataspect/softhebuttonsontheremotemadethistask____________tocomplete? a. Placement b. Color c. Labels d. Size e. Other______________________ 5. Whataspect/softheTVdisplaymadethistask____________tocomplete? a. Placement b. Color c. Labels d. Size e. Other______________________ 6. Isthereanythingyouwouldimprove? 7. Inoticedthatyouchangedthechannelusingthe[NUMBERPAD/CHANNELUPDOWN BUTTON].WastheresomethingaboutthisremotecontrolorTVdisplaythatmadeyou choosethismethod? 9|P a g e
Task3AdjustingtheVolume TaskfortheParticipant Yourfriendiscallingyoufromtheother room,butyoucan'thearhimovertheTV. Pleaseturnoffthesoundsoyoucanhear yourfriendbetter. Whenyoufeelyouhavecompletedthe taskpleaseturnofftheTV. Pleasebegin. Participantstartsthetask. Participantperformsthetaskandthen indicatesthattheyaredonebyturningoffthe TVandplacingtheremotebackonthecoffee table. ObjectivesoftheTask Completethetaskwithoutaskingfor help. Observeiftheparticipantturnedoffthe soundbyeitherthevolumecontrol buttonsorthemutebutton. Observeiftheparticipantrealizedwhere themutebuttonislocated. IssuestoExplore 1. Wastheparticipantabletocompletethe taskwithoutaskingmoderatorforhelp? 2. Whichmethoddidtheparticipantuse? a. VolumeDownButton b. MuteButton 3. Howlongdidittaketheusertoshow frustrationifany?
NotestoModerator Seeiftheparticipantlowersthevolumeusing the"volume"buttonorbypressingthemute button.
SuccessCriteria Theparticipantwasabletocompletethe task. Theparticipantdidnotaskforany assistance.
WhentoAdministerHelp Afterparticipantasksforhelponce,ask themtocontinuetoexploretheremote andTVdisplay. Iftheparticipantasksforhelpasecond time,askwheretheyarehaving difficultiesthenshowthenextstepin thepathfromwheretheyare. Iftheparticipantasksfortheinstruction manual,giveittothemtouseforthis task.

10|P a g e

Task3AdjustingtheVolume PostTaskQuestions 1. Onascaleof13howexperiencedareyouwiththistypeoftask? 1=Veryexperienced 2=Somewhatexperienced 3=Inexperienced 2. Onascaleof14pleaseindicatehoweasyordifficultthistaskwastocomplete. Circleonebasedonparticipantsansweranduseto 1=Veryeasy fillintheblanksforthefollowingquestions 2=Somewhateasy 3=Somewhatdifficult 4=Verydifficult
3. Whydoyouthinkthistaskwas____________tocomplete? 4. Whataspect/softhebuttonsontheremotemadethistask____________tocomplete? a. Placement b. Color c. Labels d. Size e. Other______________________ 5. Whataspect/softheTVdisplaymadethistask____________tocomplete? a. Placement b. Color c. Labels d. Size e. Other______________________ 6. Isthereanythingyouwouldimprove? 7. Inoticedthatyoupressedthe[MUTE/VOLUMEDOWN]button.Wastheresomething aboutthisremotecontrolorTVdisplaythatmadeyouchoosethismethod? 11|P a g e
Task4ProgrammingaFavoriteChannel TaskfortheParticipant Youalreadyknowyoulovechannel11, andyouwouldliketostoreitasyour favoritechannel. Whenyoufeelyouhavecompletedthe taskpleaseturnofftheTV. Pleasebegin. Participantstartsthetask. Participantperformsthetaskandthen indicatesthattheyaredonebyturningoffthe TVandplacingtheremotebackonthecoffee table. Objectives Seeiftheparticipantcompletesthetask usingonestraightpath. Seeiftheparticipantcompletesthetask withoutaskingforhelp. Discoverifthebuttonsandthemenuon theTVdisplayareintuitiveenoughforthe participanttounderstandhowtouse them. IssuestoExplore 1. Wastheparticipantabletocompletethe taskwithoutaskingmoderatorforhelp? 3. Whichspecificpointofthepathdidthe usershowfrustration? 4. Howlongdidittaketheusertoshow frustration? 5. Whatpathdidtheusertaketocomplete thetask?
NotestoModerator Observeiftheparticipantpressesthechannel up/downbuttonorpresses1,1.

12|P a g e

Task4ProgrammingaFavoriteChannel PostTaskQuestions 1. Onascaleof13howexperiencedareyouwiththistypeoftask? 1=Veryexperienced 2=Somewhatexperienced 3=Inexperienced 2. Onascaleof14pleaseindicateeasyordifficultthistaskwastocomplete. Circleonebasedonparticipantsansweranduseto 1=Veryeasy fillintheblanksforthefollowingquestions 2=Somewhateasy 3=Somewhatdifficult Easy/Difficult 4=Verydifficult 3. Whydoyouthinkthistaskwas____________tocomplete? 4. Whataspect/softhebuttonsontheremotemadethistask____________tocomplete? a. Placement b. Color c. Labels d. Size e. Other______________________ 5. Whataspect/softheTVdisplaymadethistask____________tocomplete? a. Placement b. Color c. Labels d. Size e. Other______________________ 6. Isthereanythingyouwouldimprove?

13|P a g e

PostTestQuestions
1. Onascaleof14pleaseindicatehoweasyordifficulttheSamsungSyncMasterP2370HD televisionwastooperate? 1=Veryeasy 2=Somewhateasy 3=Somewhatdifficult 4=Verydifficult 2. Whichtask/sdidyoufinddifficultbecauseofthedesignoftheremotecontrolandwhy? 3. Whichtask/sdidyoufinddifficultbecauseofthedesignofthetelevisionmenuandwhy? 4. Onascaleof14howlikelyisitthatyouwouldpurchasethistelevisionasifmoneywasnot afactor? 1=Iwouldbuyittoday(gotoCLOSING) 2=Iwouldthinkaboutbuyingitinthenearfuture(continuetoQuestion5) 3=Iprobablywouldnotbuyit(continuetoQuestion5) 4=Idefinitelywouldnotbuyit(continuetoQuestion5) 5. Onascaleof14howlikelyisitthatyouwouldpurchasethistelevisionifallyour suggestionswereimplementedandmoneywasnotafactor? 1=Iwouldbuyitassoonasthenewversioncameout 2=Iwouldbeinterestedinpurchasingitsoonafteritcameout 3=Iprobablywouldnotbuyit 4=Idefinitelywouldnotbuyit

Closing

Thankyouagainforparticipatinginourstudy Feelfreetohavesomemoreappetizersandbeveragesbeforeyouleave. Ifyouhaveanyquestions,feelfreetocontactSarahDulskiatsrdulski@gmail.comorby calling(312)9723136.

14|P a g e

ParticipantConsentForm
Thankyouforparticipatinginourresearch.Weareindependentresearchersconductingthis studytolearnabouttheuseoftheinteractionwiththeremotecontrolandtelevisiondisplayfor theSamsungSyncMasterP2370HD.Thistestsessionwilltakeabout4560minutesofyourtime. YouwillbeaskedtocompleteaseriesoftasksinvolvingtheSamsungremotecontroland televisiondisplaysothatwemayfindouthowtoimprovetheinteractionthatusershavewith theseinterfaces.Althoughthereisnomonetarycompensation,wewillbeprovidingappetizers andrefreshmentsattheendofthetest. Wewillberecordingyourtestsessionsothatwecanlaterreviewthetestandgatheradditional informationthatwemighthaveoverlookedduringtheinitialtesting.Allinformationgathered willbeusedonlybythepeopleconductingtheresearchandyourassociationwiththedata collectedwillremainanonymous. Therewillnotbeanyconsequencesifyoudecidetochangeyourmindatanytimeduringthis session.Ifyouhaveanyquestionsaboutthisstudy,pleasecontactSarahDulskiat srdulski@gmail.comorbycalling(312)9723136. Iagreetovoluntarilyparticipateinthisstudyandunderstandthatrecordingswillbemadeofmy session.Igranttheresearchteampermissiontousetheserecordingsforthepurposes mentionedaboveandwaivemyrighttorevieworinspecttherecordingspriortotheirusage. Pleasesignanddatebelowtoindicatethatyouunderstandtheaboveandhavehadallyour questionsanswered. _______________________________________________________________________ NAME(SIGNATURE) _______________________________________________________________________ NAME(PRINTED) ____________________________ DATE(MM/DD/YYYY)

15|P a g e