Encounters with Kismet and Cog: Children Respond to Relational Artifacts
Sherry Turkle, Cynthia Breazeal, Olivia Dast, Brian Scassellati Abstract
Kismet and Cog, humanoid robots at the MIT Artificial Intelligence Laboratory, are relational artifacts, objects designed to present themselves as having states of mind that are affected by their social interactions with human beings. Sixty children, from 8 to 13 were introduced to Kismet and Cog during the summer of 2001. The childrens first encounters with these robots provide a window onto how such objects and in particular, the robots of the future -- may enter into how children think about life, intentionality, friendship, and what is special about being a person.

Introduction

Traditionally, Artificial Intelligence largely concentrated on engineering systems that impressed through their rationality and cognitive competence whether in playing chess or giving expert advice. The past decade has seen the development of a new kind of computational object: relational artifacts. These objects present themselves as having states of mind that are affected by their interactions with human beings (Turkle 2001, 2002, 2004). They are designed to impress not so much through their smarts, but through their sociability. The humanoid robots Kismet and Cog, designed at the MIT Artificial Intelligence Laboratory, exemplify such objects, explicitly designed to relate to people in human-like ways, to detect stimuli that humans find relevant respond to stimuli in a human-like manner [and] have a roughly anthropomorphic appearance. (Scassellati 2002, 49). In the summer of 2001, Cynthia Breazeal (chief developer for Kismet), Brian Scassellati (chief developer for Cog), and Sherry Turkle (a clinical psychologist and ethnographer) all of MIT introduced a group of children to Kismet and Cog and closely observed their interactions.i This work was not experimental, but exploratory and qualitative, meant to increase understanding of the issues children raise at a first encounter with a novel form of social intelligence. This paper provides an overview of the childrens first responses, suggesting how relational artifacts engage children in powerful ways and may come to affect their views of life, sentience, and relationship.
Overview of Relational Artifacts
Relational artifacts include complex research robots such as Kismet and Cog, as well as a wider set of objects that have found their way into the consumer market: humanoid dolls, virtual creatures, and robotic pets. At varying levels of sophistication, these objects give the impression of wanting to be attended to, of wanting to have their needs satisfied, and of being gratified when they are appropriately nurtured.ii American children first met relational artifacts with the 1996 introduction of Bandais Tamagotchi, a small virtual creature whose screen is housed in egg-shaped plastic. The instruction book included in the packaging presented the child with a narrative that stressed the creatures need for nurturance: There are a total of 4 hearts on the Happy and Hunger screens and they start out empty. The more hearts that are filled, the better satisfied Tamagotchi is. You must feed or play with Tamagotchi in order to fill the empty hearts. If you keep Tamagotchi full and happy, it will Olivia Dast, Turkles research assistant, joined the project in September 2001, at the beginning of data analysis.
Encounters with Kismet and Cog/9/30/04
grow into a cute, happy cyberpet. If you neglect Tamagotchi, it will grow into an unattractive alien.iii The Tamagotchi requires that its user determine whether it needs to be cleaned, fed, or amused by assessing its state on a small screen display. If the user, usually a child under 12, successfully reads and responds to the digital creatures state of mind, the toy will be happy. It will flourish and survive. In the toys that were produced for the Japanese market, the penalty for not caring adequately for a Tamagotchi is the creatures death. In some versions, the dead Tamagotchi could be uploaded to a virtual graveyard. In the United States, manufacturers decided on a less harsh resolution: a neglected Tamagotchi becomes an angel and is uploaded to its home planet. Additionally, in the United States version, a user can hit the reset button and be presented with another creature. Even with the opportunity for multiple chances, from the childs point of view, acceptance of the caretaker role is the crucial step in creating a bond with the inanimate; Bandais website provided clear moral instruction to link responsibility with nurturance in the relationship with a virtual pet: Tamagotchi is a tiny pet from cyberspace that needs your love to survive and grow. If you take good care of your Tamagotchi pet, it will slowly grow bigger, healthier, and more beautiful every day. But if you neglect your little cyber creature, your Tamagotchi may grow up to be mean or ugly. How old will your Tamagotchi be when it returns to its home planet? What kind of virtual caretaker will you be?iv Furbies, the toy fad of 1998-99, are small furry creatures with large, prominent eyes and the ability to speak. In the case of Furbies, the childs caretaking responsibility is centered on teaching. Like Tamagotchis, Furbies are presented as visitors from another planet. This explains why they only speak Furbish when they are first brought to life: it is the mother language of their planet. In the course of play, Furbies learn to speak English. In fact, this learning reflects the unfolding of a program that evolves Furby language to a set of simple English phrases. (In other words, no matter what language a child speaks to a Furby, that Furby will learn English.) For most children 5-9, the illusion works: children believe that they are teaching their Furby by interacting with it. As in the case of Tamagotchis, Furbies demand attention; children understand that a lack of attention will have a negative impact on the toys inner state. Similarly, My Real Baby, introduced by Hasbro in 2000, has inner states that a child needs to decipher (through its baby sounds and baby facial expressions) in order to appropriately nurture the toy. My Real Baby was a descendent of a robotic doll first developed by iRobot, known as Bit. Rodney Brooks, the Director of the MIT Artificial Intelligence Laboratory and founder and director of iRobot, describes Bit in terms of its inner states: If the baby were upset, it would stay upset until someone soothed it or it finally fell asleep after minutes of heartrending crying and fussing. If Bitwas abused in any way- for instance, by being swung upside down- it got very upset. If it was upset and someone bounced it on their knee, it got more upset, but if the same thing happened when it was happy, it got more and more excited, giggling and laughing, until eventually it got overtired and started to get upset. If it were hungry, it would stay hungry until it was fed. It acted a lot like a real baby. (Brooks 2002, 109) Although more sophisticated (and more expensive), Sonys Aibo home entertainment robot (in the shape of a dog) participates in the basic narrative of connection through caretaking that characterizes Tamagotchis, Furbies, and My Real Babies. Aibo responds to noises, makes musical sounds to communicate and express different needs and emotions, and has a variety of sensors that respond to touch and orientation. The robot dog develops different personalities depending on how it is treated by its user. Newer Aibo models have facial and voice recognition software that enable the creature to recognize its primary caregiver. Cog and Kismet are highly evolved examples of relational artifacts. Cog is an upper-torso humanoid robot, with visual, tactile, and kinesthetic sensory systems. Cog is capable of a variety of social tasks including

visually detecting people and salient objects, orienting to visual targets, pointing to visual targets, differentiating between animate and inanimate movement, and performing simple tasks of imitation. Kismet is a robotic head with five degrees of freedom, an active vision platform, and fourteen degrees of freedom in its display of facial expressions. Though the heads sits disembodied on a platform, it is winsome in appearance. It possesses small, mobile ears made of folded paper, mobile lips made from red rubber tubing, and heavily lidded doll eyes ringed with false eyelashes. Its behaviors and capabilities are modeled on those of a pre-verbal infant. (Breazeal and Scassellati 1999, 2000; Breazeal 2002, 5). It gives the impression of looking into peoples eyes, can recognize and generate speech and speech patterns, albeit to a limited degree.v Kismet engaged children through its ability to make eye contact, to exhibit facial expressions, to display affective states, and to respond to language with utterances of its own, including the ability to repeat a requested word, most often Kismet. Cog engaged children by looking in their direction and copying their arm motions. Kismet engaged children through its ability to make eye contact, to exhibit facial expressions, to display affective states, and to respond to language with utterances of its own, including the ability to repeat a requested word, most often Kismet. Cog engaged children by looking in their direction and copying their arm motions.

The Study

A major tradition of Human Robot Interaction studies are aimed toward the production of new machines that are able to hold the attention and interest of the user while engaging in effective (task-oriented) social interactions. (Bourke and Duffy 2003; Baltus et al. 2001; Breazeal 2002; DiSalvo and Gemperle 2003; Goetz and Kiesler 2002; Kidd 2003). The focus of this report is rather on the fantasies and expressive behavior of the children, in particular on the question of how children situated themselves in social relationships with the robots. What were the children able to express about their sense of these relationships and their meaning, even through brief first encounters? Participants in our study, from 8-13, all of whose identities are disguised, were drawn from summer afterschool programs and community organizations, such as a performing arts summer camp. The research team posted invitations to participate in a study about what children think about robots; families interested in the study initiated the contact. Participants represented a wide range of socio-economic positions, and ethnicities.vi Upon arriving at MIT, each child was assigned to a clinical researcher who accompanied him or her through the day. At least one clinical researcher and one roboticist staffed each encounter between child and robot. Interactions with the robots were video recorded and a combination of audio and videotapes were used to document the remainder of the childs time in the laboratory. During the one-onone visits with the robot, participants were told that they could do whatever they wanted as long as it was not harmful or dangerous either to themselves or the robot. Participants were asked to wear a wireless clipon microphone, which the clinical researchers explained was being used to assist in recording their conversation in the noisy laboratory room. In addition, Kismet actually used this audio signal to detect word choice and vocal prosody. Cog did not use this information. Our approach was to interfere as little as possible with each childs encounter with the robot. If necessary, participants were assured that they could approach and touch the robot. If they asked for guidance or seemed anxious, the research team provided a supportive question (i.e., What do you notice about Kismet right now?) or, if necessary, a more directive request (i.e. Can you try to get Cog to wave at you?). Stuffed animals were available in the areas around both robots and children were told that they could use them if they wished. In general, each child had about 20 minutes alone with the robot and was told when they had five minutes remaining in their session. After the session with the robot, children had a conversation with their assigned clinical researcher about their experience. After this interview, children came back for a group session of about 30 minutes with the robot and the roboticist during which they were encouraged to chat with each other, and to ask questions. Each child spent about 50 minutes with the robot. Most interacted with only one robot. vii Sessions with Cog included a special debriefing during which Scassellati explained the mechanics of the robot to the children. Each child then was provided the opportunity to drive Cog, which meant turning

off the robot as an autonomous actor and assuming control of its behavior. This explicit debriefing presented a chance to test the limits of the childrens perseverance in their animation of the robot. Indeed, one of our most striking findings is that children persevered in their animation and anthropomorphization of these humanoid robots even when the robots failed to operate properly and even when there was a determined effort to demystify the machine. Children continued to imbue the robots with life even when being shown -- as in the famous scene from the Wizard of Oz -- the man behind the curtain. Here we describe three major themes that emerged from our study of first encounters. Children display perseverance in their efforts to communicate with the robots, including finding ways to explain and excuse the robots failures to communicate with them, perseverance that expresses a range of personal styles. A similar range of styles marks childrens ways of anthropomorphizing the robots. For the most part, children come to see the robots as sort of alive because they feel in a social relationship with them and use a range of strategies to overcome disappointments and system failures. Finally, childrens stake in preserving a sense of relationship is so strong that they actively resist any demystification of the robots. With few exceptions, children were uninterested to the point of unwilling to understand the robots in terms of underlying mechanism.
Theme 1: Perseverance and the expression of personal style: Robot as Rorschach
Kismet and Cog are research robots in a university laboratory environment. In the course of our study, laboratory members were continually working on them. The goal of this work is improving robot performance. Its cost is an unstable platform. At various points during our study, Kismet had difficulties tracking eye movement, responding to auditory input, or vocalizing. And one of Cogs arms was inoperable for the duration of the study, limiting its abilities to imitate motion. Despite these limitations, children persisted in trying to elicit speech from Kismet (with the greatest focus on getting Kismet to say its name) or getting Cog to imitate their arm movements. Heather, 7, is energetic and vibrant. She is small and thin, with sparkling blue eyes, and a ready grin. Before meeting Cog, Heather informs the clinical researchers that if she could take a robot home she would treat it just like a pet. It would sleep outside and she would give it bones to eat. During her session, Heather speaks to the robot directly. Scassellati introduces Cog and demonstrates that the robot can raise its arm by imitating human movement. When she is alone with Cog, Heather performs what she calls an experiment where she tries to have the robot raise its arm to model her pointing gesture then attempts to place a stuffed animal on Cogs raised arm.viii Heathers goal is to have Cog balance the toy on its raised arm. With each attempt, Heather raises her arm and instructs Cog to do the same. In a typical interchange, Heather says: Up. Up. Up. Like you are pointing at me. Up! now steady. and rushes to place the toy on Cogs lifted arm. Each time she succeeds in balancing the toy, Cogs arm drops. Undeterred, Heather always tries again: Now lets try that again. Up. Up, robot. Uuuuuuuuup! Thank you. Now steady. Heathers tone ranges from commanding to pleading as she tries a range of toys in the seeming hope that one or another toy might better hold the robots attention. She refuses to give up and continues her experiment until the very end of her individual session. (Session 17, S52)

Kismet is silent. Trisha pets the space between Kismets eyes and says, Dont be scared. Before she leaves, Trisha gives Kismet a hug. (Session 10, S30) Children related to both Cog and Kismet as sentient, but most children saw Cog as a playmate or peer while Kismet tended to evoke the desire to parent. It was common for children to say that Kismet reminded them of an infant or younger sibling. When children played with Kismet, they often urged the robot to pay attention to them, to listen, to try harder. Mandi, 9, radiates energy. She tells the clinical researchers that what she enjoys most is bothering her three older siblings and playing with her baby sister. She first asks Kismet, What is your name? and subsequently, Do you have parents? Mandi is unusually gentle with Kismet. Encouraged by a vocal response from Kismet, she asks, Do you have brothers and sisters? Since there is no answer, she repeats her question three times. As the session unfolds, Mandi increasingly speaks of Kismet as a female, using she and her and seems to treat Kismet as a little girl. Mandi asks Kismet, Do you like ice-cream? (Kismet says something) I think she said yes. What kind of flavor do you like? Mandi cycles through different conversation topics, asking Kismet what her favorite color is, if she goes to school, if she has any toys. Mandi plays tries to engage Kismet with its toys, dangling them for Kismets enjoyment. Mandi seems to analogize Kismet to her six-week-old sister. Speaking to a clinical researcher, she says, I think that she [Kismet] is a baby because these toys look like little baby toys. Mandi says that Kismet is a little bit older than her sister because Kismet speaks better. Further, Mandi says that Kismet might get sad just like a baby even if she cant cry and that if the roboticists made arms for Kismet that she would scribble over the pages of a coloring book and put things in its mouth. She thinks Kismet has a birthday and that the robot was born from a stomach. In her post interview, Mandi maintains her belief that Kismet is born and that she has parents, as well as brothers and sisters. Mandi believes that Kismet learns to speak better with every child that comes to visit. She says that [Kismet] is still little, but it grows up. It looks little, but it still grows up. that she will continue to learn and will mature inside even if we might not be able to tell from outside. (Session 16, S49) Many children believed that they had taught Kismet something. The act of teaching, a form of nurturance, reinforced the bond between child and robot. It also gave children the sense that Kismet thinks, remembers, and is sometimes in more of a mood to learn than at other times. The confidence Mandi expressed in Kismets internality, a common assumption, supported her identification with the robots learning process. To children, Kismet was like them and although it was having difficulty, it seemed to be trying. Some children were pleased that Kismet trusted them enough to learn from them. Children were openly affectionate with Kismet, showering it with hugs, kisses, and caresses, making sustained efforts to entertain it with stuffed animals and rattles. Some tried to amuse it with favorite childhood games such as Peek-a-Boo and favorite childhood songs. In one case, a child made clay treats for Kismet to eat. Poignantly, one child told Kismet that he was going to take care of it and protect it against all evil.

Perseverance through belligerence
Some children showed no less determination to stick with Kismet and Cog in spite of the robots frustrating behavior, but persisted with anger rather than nurturance. Two boys exemplified this style of engagement. Adam, 6, has curly chestnut hair. He is rather small but seems to have twice the energy for his size. Andrew is very articulate and seems competitive, and mischievous. Before the beginning of the session, Adams father tells the clinical researchers that Andrew had two questions: Could he take the robot home afterwards, and did it have weapons? Adams father
says that his son has been into fighting games, likes rough playing, and likes to be in charge at home and school. In his initial meeting with Kismet, Adam asks the robot, Can you talk? When Kismet doesnt answer, Adam repeats his question loudly and with increased urgency. Adam becomes very frustrated when Kismet persists in staring into space. Adam tries to understand Kismet, but soon decides that it doesnt make any sense. After a series of What? What? What? Adam tells Kismet to shut up! Adam starts forcing various objects in Kismets mouth, first a metal object then a toy caterpillar saying, Chew this! He becomes increasingly angry at Kismet for not paying attention to him and for not being comprehensible. At no point does he disengage from the robot. (Session 27, S27)
Jerome, 12, was born in France and moved to the States when he was four. He visits the robot lab with his two younger brothers and begins his time there as very unenthusiastic about the enterprise. He reluctantly answers the clinical researchers questions and speaks through harsh insults to his brothers, compounded with aggressive behavior towards them. He asks Kismet half-heartedly, Whats your name? When he does not receive an answer, Jerome covers Kismets cameras and orders, Say something! After a few more minutes of silence he then shouts, Say shut up! Say shut up! Seeming to fear reprimand, Jerome continues with less hostile words, but continues with his brusque tone: Say hi! and Say blah! Suddenly, Kismet says Hi back to him. Jerome, smiling, tries to get Kismet to speak again but when Kismet does not respond he forces his pen in Kismets mouth and says, Here! Eat this pen! Though frustrated, Jerome does not tire of the exercise. (Session 20, S58)
Perseverance through resourcefulness
Robot unresponsiveness served as a window onto childrens various styles of coping with frustration. Some nurtured; some cajoled; others reached for alternate means of communication, assuming always that the robots were alive enough that their failures to communicate could be overcome through increased effort. So, for example, some children tried to speak Kismet back to Kismet, repeating the babble they heard from the robot, while others acted hurt and tried to make Kismet feel guilty about not speaking to them. Others tried speaking foreign languages with Kismet, interpreting its difficulties as those that might be encountered by any alien. Roanne, 12, is very eager to speak with Kismet, but the robot is not answering her questions. Roanne patiently tries to engage Kismet. She asks, Do you sing? Do you sing? {slower, more articulated speech} Do you sing? Say yes I think he speaks French. Do you sing? Do you sing? [Kismet speaks] He said he trusts me! Ok! When it is time to end the session, Roanne says Adios to Kismet. Roanne decides that Kismet spoke to her in Spanish. When asked what she thought Kismet might be saying, Roanne replies, All it said was, I cant remember, but he said get lost stuff like that, and I cant remember the other words. The other words he said in Spanish. The clinical researcher then asks, Do you think when you said Adios it understood that? Roanne answers, Yeah. (S25, Session 8)

Perseverance through the ELIZA effect
Children want the robots to be responsive. When the robots were not working perfectly and could not comply, children go to great lengths to cover for them and their limitations. Even when they were told that the robot with which they were playing was broken or that a particular function was not working (in other words, when they were given mechanistic explanations for robot problems) children created explanations that preserved their image of the robot as sentient and caring about its relationship with them. The tendency to work around a computers relational limitations has long been part of our understanding of computer-human interaction. Joseph Weizenbaums ELIZA (or Doctor) program was designed to respond in the manner of a Rogerian psychotherapist (it mirrored a statement Im angry at my mother and turned it into a solicitous reply: Why do you say you are angry at your mother?). The program was seductive, even Weizenbaums graduate students who knew that the program could not know or understand wanted to converse with it and confide in it. They wanted to be alone with it (Weizenbaum 1976). Weizenbaum himself became indignant, insisting that he had written the program as a joke and was troubled that people wanted to converse seriously with what he estimated to be little more than a parlor trick. In her studies of peoples relationships with ELIZA, Turkle observed that people helped ELIZA to seem more intelligent than it actually was. They refrained from making comments or asking questions that might have confused it and asked it questions that would ensure a human-like answer, going to considerable ends to protect their illusion of a relationship with it (Turkle 1984, 40). In The Second Self, Turkle describes a 5year-old named Lucy who creates dialogues to fit the Speak and Spells speaking abilities and maintain an illusion of conversation. Lucy would tailor her demands to what the toy could say; she could essentially force the toy to address her. We saw many examples of this helping behavior in our study, particularly in relation to Kismet. When all evidence pointed to a broken or malfunctioning robot, children rationalized
Kismets failings: the robot is deaf, it is too young to understand and respond correctly, it is ill, it is not responding because he doesnt like me, it makes sense but is speaking another language, it is very shy, and it is not broken but sleeping. What was at stake was not just an image of Cog and Kismet as intelligent and intentional beings. More centrally, childrens excuses and helping behavior preserved their sense that the robots cared about them. Jonathan, 8 is terrorized by his two older brothers whose favorite pastime is to beat him up. In contrast to his brothers, Jonathan is talkative, light-hearted, and displays much enthusiasm about being at the lab. He wears an oversized blue tee shirt and shorts. He tells the clinical researchers he wishes he could build a robot to save him from his brothers. He says that he wishes he could have a robot as a friend to tell his secrets to and to confide in. Jonathan says that he is sure Kismet will talk to him. Upon meeting Kismet, he tells the robot, Youre cool! As Kismet vocalizes, Jonathan interprets Kismet as saying what he wants, first What are you doing, Harry [one of his brothers]? and then Im going to kiss you. He said, Im going to kiss you! Kismet continues incoherently babbling. Jonathan nevertheless smiles and says again, Youre cool! with a thumbs-up. Though Kismet is not actually repeating Jonathans suggested words, Jonathan turns to the researchers and says See! It said cheese! It said potato! Jonathan says that Kismet is learning and is saying the words he is teaching it to say. At other times, Jonathan would make explanations for Kismets incoherence. For example, when Jonathan presents the dinosaur toy to Kismet and it utters something like Derksherk Jonathan says, Derksherk? Oh he probably named it [the toy]! Or maybe he meant Dino, because he probably cant say dinosaur. When Kismet stops talking completely, Jonathan suggests Maybe after a while he gets bored. He tries to use Kismets toy to get its attention, when this fails he says that the toy actually distracts Kismet. Jonathan is showed Kismets voice recognition display, which displays what Kismet is hearing. Jonathan tries to speak Kismet language, repeating what he sees on the monitor. When this doesnt work to prompt a response from Kismet he insists, I dont think its hearing me so good. Towards the end of the session, Jonathan concludes that Kismet stopped talking to him because it liked his brothers better. (Session 20, S60)

These laboratory findings are consistent with Turkles ethnographic and clinical findings about anthropmorphization in childrens interactions with computational objects. Turkle began working with children and computational objects in the late 1970s. The first objects of her investigation were the first generation of computer toys and games: Merlin, Simon, Speak and Spell (1984). In those cases, even a minimum of interactivity and reactivity provoked children to imagine them as sentient others. Therefore, with robots such as Kismet and Cog (which can draw on the power of the gaze and specifically designed to be actively relational) it would be surprising if children did not imbue them with
human-like traits. What was striking, however, was the range of styles of anthropomorphization that was enabled by the robots sociable design.
Anthropomorphization through extravagance of detail
One indication of the degree to which children anthropomorphized the robots was that children almost always described the robots as gendered. Turkle notes that earlier computational objects were sometimes treated as gendered, usually referred to as he, but that many children used it as a default unless asked a question about gender, or moved back and forth between he and it or between she and it. In the present study, there was no such slippage. Cog was usually regarded as an adult male and Kismet as a female child, but some children thought both robots were males. Additionally, the machines were consistently thought to be emotional as well as intelligent. Children asked Cog and Kismet how they were feeling, if they were happy, did they like their toys? Children asked if the robots loved them. Children spoke about the robots as if they were persons with minds and feelings. Fara, 11, has short chestnut hair and seems particularly mature for her age. Her father is from Egypt and has home-schooled both Fara and her younger brother. She is one of the few children who spends time with both robots. Upon seeing Cog, her first reaction is to exclaim, Oh, its so cute! when asked to comment on this, Fara says, He has such innocent eyes, and a soft-looking face and that this contrast with Cogs masculine body. When Cog raises its arm spontaneously, she exclaims, I wonder what hes thinking. Fara is direct in her efforts to engage Cog. She asks, What do you want? What do you like? and then, tries to startle Cog. She sneaks up to the robot, jumps out loudly, and noticing Cogs lack of reaction, says to him, You dont get scared, do you?

Anthropomorphization through Mutuality
Relational artifacts evoke a sense that the user and the artifact are in a relationship. Children not only see these objects as alive and sentient, they believe that their feelings for the objects are returned. In studying previous generations of computational objects, Turkle has described the computer as a Rorschach, as a relatively neutral screen onto which people are able to project their thoughts and feelings, a mirror of mind and self But todays relational artifacts make the Rorschach metaphor far less useful. The computational object is no longer affectively neutral. Relational artifacts do not so much invite projection as demand engagement. People are learning to interact with computers through conversation and gesture. People are learning that to relate successfully to a computer it is not necessary to know how it works. Rather it is possible to take it at interface value, to assess its emotional state, much as they would were they relating to another person (Turkle 1984, 2002). In the past, the power of computational objects to act as relatively neutral screens meant that children could project their own meanings onto them. Relational artifacts take a more active stance. With them, childrens expectations that their computational objects want to be hugged, amused, or loved dont only come from the childs projection of fantasy or desire onto inert playthings, but from such things as a digital dolls crying inconsolably or even saying: Hug me! or Its time for me to get dressed for school! Such behavior inhibits projection -- something that will have significant implications for the kinds of
satisfactions and developmental growth that children can obtain from playing with robots but unquestionably increases a childs sense that he or she is in a specific relationship. The psychology of engagement over the psychology of projection was apparent even in childrens responses to Furbies, which simulate learning and loving in very limited ways. In following childrens responses to Furbies, Turkle found that childrens ideas about their affection for Furbies became enmeshed with fantasies about how Furbies might be emotionally attached to them (Turkle 2001). When Turkle asked children, Do you think the Furby is alive? their responses were not in terms of what the Furby can do, but how they feel about the Furby and how the Furby might feel about them. Charlie (age 6): Well, the Furby is alive for a Furby. And you know, something this smart should have arms. It might want to pick up something or to hug me. Beyond being smart and alive, relational artifacts evoke love and friendship. In our summer 2001 study, children spoke directly about experiencing mutual affection and connection with Cog and Kismet. They also signaled their experience of mutuality by unprompted expressions of affection, and the importance of being recognized, acknowledged, and liked by the robots. Children spontaneously kissed Kismet and hugged Cog. Children sang to them and put on dance shows. When Kismet successfully said one of the childrens names, even the oldest and most skeptical children commented that this was proof that Kismet liked them. Likewise, if Kismet said the name of another child when one was trying to get Kismet to say their name, this was taken as evidence that Kismet preferred the other child, often causing hurt feelings as well as lively disputes about who Kismet really preferred. When either Cog or Kismet were unresponsive, this was taken as proof that the robots did not like them. The children did not experience a broken mechanism but a personal rejection. Jazmyn, 9, is wearing a bright multi-colored tank top over a white tee shirt. She is bubbly, very friendly and enthusiastic. Jazmyns favorite extracurricular activity is dance; she says that she takes step, hip-hop, jazz, and Latin dance classes. Jazmyn immediately connects with Cog. Scassellati is there with her and they have the following dialogue: Scassellati: What have you seen the robot do so far? Jazmyn: Look at me and raise his hands. Maybe its trying to shake my hand or something. Scassellati: Did you try shaking its hand? Jazmyn: Yeah Scassellati: What did it do? Jazmyn: Shook my hand. Later, Jazmyn asks if the roboticists were planning on making a mouth for Cog. She says that Cog probably wants to talk to other people and it might want to smile. Jazmyn expresses her desire for Cog to relate, to reach out to make contact with her; this is similar to Charlies wishing that the Furby would hug him back. As Jazmyn continues her conversation with Scassellati, there is another striking moment. Scassellati: What do you think the new version of Cog should be able to do? Jazmyn: Dance. Scassellati: Should it just dance for you or should it be able to dance with you? Jazmyn: Dance with it! Scassellati: Do you want to dance with the robot? Jazmyn: Yeah! Scassellati: What kind of dancing would you do? Jazmyn: Any kind! Speaking to one of the clinical researchers, Jazmyn looks pensive and says, If his other arm could move, I think that I would teach him to hug me. Jazmyn talks about her desire to take Cog home where they would play together, dance together, talk together, and eat dinner

together. Later she dances for Cog and says that Cog really enjoyed her performance and thought that it was very interesting. She adds, I liked that the robot was at first by itself and that it looked at me when I was dancing. I liked that the little circles and squares followed movement while I was moving. (Session 13, S41)
Eugene, 8, is thin with large friendly eyes and a ready grin. His mother informs the clinical researchers that Eugene has been playing with LEGOS since he was 2 and that they are his favorite toys. Eugene wanted to bring his remote control car to the robot session so he could ask the engineers at MIT how it works. Eugene is visibly excited in the anticipation of meeting Kismet. Kismet is temporarily broken, and so the roboticist explains how Kismet works. Eugene asks many questions: what is Kismet made of, how does it work, and how long did it take to build the robot. When told that the robot is called Kismet, Eugene responds that this is an unusual name for a child. He seems quite surprised when the roboticist tells him that Kismets parts are made and not grown. Eugene thinks that Kismets cables are its hair. When he encounters the robot, Eugene sees Kismet as a male. Eugene says that Kismet is not broken but just sleeping hes sleeping with his eyes open just like my dad does. Eugene places his arm around Kismet and declares, He will make a good friend. Eugene says he can tell Kismet will be a good friend because of its smile. He says that Kismet looks like a normal good friend. Before leaving Kismet for the first time, he hugs it and tells Kismet he will see him soon. Eugene and his sister, 6, are asked what they would do with Kismet. Eugene says, play baseball and eat ice-cream together, his sister simply says, Love it. Eugene says that Kismet would enjoy playing video games and would probably beat him. Eugene and his little sister extensively hug and kiss Kismet and tell the robot how much they love him. Eugenes last few minutes with the robot are especially moving. Kismet is babbling nonsense syllables and Eugene is trying to teach Kismet, to have him speak, Say I love you. Kismet, say I love you. Eugene tries again and again with a kind voice. Kismet is silent and Eugene looks like he could cry. A few minutes later, Kismet speaks and Eugene decides with what can only be called intense relief, He said I love you. Eugene and his sister then proceed to try to have Kismet say their name and are overjoyed when they hear Kismet utter their name. It is time for them to leave, the children look sad. Eugene hugs Kismet gently, his sister kisses Kismet. Eugene looks at Kismet as if indeed he was leaving a friend, and caresses him one more time before a final embrace. (Session 18, S53)

Another sign of mutuality in childrens sense of the relationship was childrens pleasure when Kismet succeeded in speaking or Cog finally raised its arm, a response that exceeded expressions of pleasure children always have when they get a toy to work. Children reflected on Kismet and Cogs having made an effort, or having performed a job well done. This pleasure was more akin to parental pride. As children assumed the parental role, they made it clear that their encouragement had been decisive for the robots. When a robot succeeded, children took this as evidence that their patience had borne fruit or that a particular learning strategy had worked. They made the robots success into their success.
Theme III. Resistance to Demystification
Even in the very brief encounters with robots, children were drawn into relationships that seemed to matter to them, relationships in which the robots behavior affected their state of mind and self-esteem. Our team explicitly discussed the ethics of such encounters and the possibility that the development of such feelings
should be tempered by presenting children with a realistic or engineering perspective on the robots behavior. We found, however, that childrens sense that the robots were alive, the degree to which they anthropomorphized them, and their sense of consequence from the interactions with the robots were little if at all affected by learning more about how the robots worked, i.e. their mechanical nature. In our fieldwork, we explicitly tried to experiment with the effects of methodically demystifying the robots, with the idea that such honesty might make the relationship between person and machine more authentic. We used strategies for making the robot transparent to the users (in terms of underlying mechanism) with both Kismet and Cog, but were most systematic in addressing this issue with Cog. Scassellati had a particular interest in developing responsible pedagogy in robotics. He was committed to showing children the machine behind the magic, feeling that it was inappropriate for children to leave the laboratory under the illusion that Cog was an animate creature. In our study, 30 children first had an individual play session with Cog, then were joined by Scassellati who proceeded to explain to the children exactly how Cog worked, giving a real-time demonstration of how it processes information. Children were shown the computers that helped to run Cog, the monitors that showed what Cog saw, and were encouraged to ask any questions they might have about how the robot functioned. Finally, and most importantly, children were allowed to drive the robot, meaning that they had a chance to control the robots movements and behaviors. Metaphorically, they got to see the robot naked. When Scassellati first suggested giving children a reality check by putting them in touch with the robot as a transparent mechanism, there was much discussion about its possible impact. Researchers hypothesized that the demonstration would alter childrens sense of Cog as sentient. Turkle has studied peoples relationships with increasingly opaque computers and charted a trajectory from a culture of calculation (based on the idea that the computer could be understood in mechanical terms) to a culture of simulation (here, computers have become objects that present as opaque and people are asked to take them at interface value (Turkle, 1995, 1997). As the focus shifted to interaction with an opaque surface, people increasingly related to the machine as a psychological entity. Now, we were intrigued by the possible effects of didactically insisting on a transparent view of an otherwise opaque robot. Would the robot, now presented as mechanical, systematically stripped of its extraordinary powers, and more relevantly, of any illusion of its autonomy, seem less likely to serve as a companion, seem less worthy of relationship? In the event, stripping the robot of its powers and making it transparent had very little visible effect.ix It seemed akin to informing a child that their best friends mind is made up of electrical impulses and chemical reactions. Such explanations (on a radically different level from the one at which relationships take place) are treated as perhaps accurate but irrelevant to ongoing relationship. They might be helpful in explaining a friends bad mood just as Scasselattis debriefing might be helpful in explaining why Cog might be having a bad day. It was not that the explanation was not welcome; it was received as interesting, some children even found it compelling. But it did not interfere with the sense of relationship. The result here is similar to what we observed when the Kismet and Cog malfunctioned during a play session. At those times, children did not treat the robots as broken mechanisms but as ailing social creatures. It seems that once children understood that these particular robots were capable of sociability, the machines were treated as social creatures no matter what their immediate state. Once defined as social, any lack of particular competencies is treated as an unfortunate disability for which the robot deserves empathy. The following vignette exemplifies the irrelevance of understanding the mystery behind the machine for most children: Blair, 9, wears a red tee shirt and shorts to the session; she is tall and looks slightly older than her age. She is the most assertive, and at times, defensive and aggressive of the group of children participating in the session. She says she believes that Cog can be her friend and says that taking it home would be like having a sleepover! She also believes that robots, like people, can dream and have nightmares. When asked if she thinks Cog has feelings she says that she thinks it certainly does and adds, maybe he was shy because he kept putting his arm out to us

appeared in the role of nearest neighbor people were distinguished by what made them different from the machines. Through the mid-1990s, in large measure, children made these comparisons between computers and people by focusing on what computers could do. In contrast, in the company of relational artifacts, childrens conversations about what makes people special contrasted the relational competencies of computers and people. It became a discourse not about technical competencies but about relational competencies (Turkle 1984, 1990, 2000, 2004). Among the ideas about relational competencies that came to the foreground is the notion that humans are special because of their imperfections. This study of first encounters offered poignant testimony to the way that a certain vulnerability and frailty become valued as a defining trait for people. A ten-year-old who has just played with Kismet says, I would love to have a robot at home. It would be such a good friend. But it couldnt be a best friend. It might know everything but I dont. So it wouldnt be a best friend. She further explains that a robot is too perfect and that it might always need to correct her. Friendship is easier with your own kind. In the end, our look at first encounters between children and relational artifacts left us with confidence that the future of human-computer interaction will have children seeing robots as alive (at least in their way), and feeling an emotional as well as intellectual connection with them. The children and computers will be in relationship, but the self reflection evoked by these involvements invite new and complex questions, perhaps most centrally, What is a relationship?

Resources

G. BALTUS, D. FOX, F. GEMPERLE, J. GOETZ, T. HIRCH, D. MAGARITIS, M. MONTEMERLO, J. PINEAU, N. ROY, J. SCHULTE, S. THRUN (2001). Towards Personal Service Robots for the Elderly. In Proceedings of the Workshop on Interactive Robotics and Entertainment (WIRE). Computer Science and Robotics, Carnegie Mellon University. J. BOURKE, B.R. DUFFY (2003). Emotion Machines: Projective Intelligence and Emotion in Robotics, IEEE Systems, Man & Cybernetics Workshop (UK&ROI Chapter), September. C. BREAZEAL and A. FOERST (1999). Schmoozing with Robots: Exploring the Boundary of the Original Wireless Network. Presentation at the Third International Conference on Cognitive Technology (CT August 1999). C. BREAZEAL, B. SCASSELLATI (1999). How to Build Robots that Make Friends and Influence People, presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-99), Kyongju, Korea. C. BREAZEAL (2000). Sociable Machines: Expressive Social Exchange Between Humans and Robots. MIT Ph.D. Dissertation, MIT. Department of Electrical Engineering and Computer Science. C. BREAZEAL, B. SCASSELLATI (2000). Infant-like Social Interactions Between a Robot and a Human Caretaker, Adaptive Behavior. 8(1). C. BREAZEAL (2002). Designing Sociable Robots. Cambridge: MIT Press. R. BROOKS (2002). Flesh and Machines: How Robots Will Change Us. New York: Pantheon Books. C. DISALVO, F. GEMPERLE (2003). From Seduction to Fulfillment: the Use of Anthropomorphic Form in Design. Proceedings of DPPI '03 : Designing Pleasurable Products and Interfaces, Pittsburgh, PA. ACM press. J. GOETZ, S. KIESLER (2002). Cooperation with a Robotic Assistant. In Conference on Human Factors In Computing Systems (CHI 2002), (Minneapolis, MN, USA). C. KIDD (2003). Sociable Robots: The Role of Presence and Task in Human-Robot Interaction. Master's Thesis, MIT. Program in Media Arts and Sciences.

S. KIESLER, L. SPROULL (1997). Social Responses to Social Computers, in B. Friedman, Human Values and the Design of Technology, CLSI Publications. C. NASS, Y. MOON, J. MORKES, E-Y KIM, B.J. FOGG (1997). Computers are Social Actors: A Review of Current Research in B. Friedman (Ed.) Human Values and the Design of Computer Technology. Stanford, CA: CSLI Press. J. PIAGET (1960). The Childs Conception of the World. trans. by Joan and Andrew Tomlinson. Totowa: Littlefield, Adams. R. PICARD (1997). Affective Computing. Cambridge: MIT Press. S. PARISE, S. KIESLER, L. SPROULL, K. WATERS (1999). Cooperating with Lifelike Interface Agents, Computers in Human Behavior, 15, 123-142 B. REEVES, C. NASS (1999). The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge: Cambridge University Press. B. SCASSELLATI (2002). Foundations for a Theory of Mind for a Humanoid Robot, Ph.D. Dissertation, MIT. Department of Computer Science and Electrical Engineering. L. SPROULL, R. SUBRAMANI, S. KIESLER, J. WALKER, K. WATERS (1996). When the Interface is a Face Human-Computer Interaction, 11, 97-124 (Reprinted in Friedman (1997) Human Values and the Design of Technology, CLSI Publications). S. TURKLE (1984). The Second Self: Computers and the Human Spirit. New York: Simon and Schuster. S. TURKLE (1995). Life on the Screen: Identity in the Age of the Internet. New York: Simon and Schuster. S. TURKLE (1997). Seeing Through Computers: Education in a Culture of Simulation. The American Prospect, no. 31, March-April. S. TURKLE (2001). Relational Artifacts. Proposal to the National Science Foundation SES-01115668. S. TURKLE (2002). Whither Psychoanalysis in the Computer Culture. Bulletin of the Freud Museum. S. TURKLE (2004) Relational Artifacts. Final Report on Proposal to the National Science Foundation SES-01115668. J. WEIZENBAUM (1976). Computer Power and Human Reason: From Judgment to Calculation. San Francisco: W. H. Freeman.
This research was funded by a NSF ITR grant Relational Artifacts, (Turkle, 2001) award number SES-0115668 and by the Mitchell Kapor Foundation through its support of the MIT Initiative on Technology and Self. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation nor of the Mitchell Kapor Foundation. Breazeal, Scasselatti and Turkle participated in data collection, discussion of the findings, and oversight of the summer project; Turkle and Dast took responsibility for the analysis of the data and redaction of this text. Study participants also include Jen Audley, research coordinator during summer 2001, and research assistants Robert Briscoe, Anita Chan, Tamara Knutsen, and Rebecca Hurwitz. ii Indeed, at MIT, there is an affective computing, research group that undertakes to develop computers capable of assessing their users emotional states and of responding, in turn, with