I N T E R A C T I V E TA B L E
S T E F F E N S T R T Z , A N D R E A S M A G N U S S O N , M A R K O S K A S S A G O B E N A , A B D U L R A H I M N I Z A M A N I , V I J A YA M A D H A V A R E D D Y D A G G U M AT I

20.12.2008

This project was carried out to fulll the requirement of the course Ubiquitous Computing 2008. The "Interactive Table" is an interactive table with a simple and intuitive tangible user interface which immediately reacts to the objects which are placed on it. The design does not consider a specic use of the table, rather to be a general table, with its height between a coffee table and a dining table used in a home environment. Keywords: Multi-touch, Table, Interactive, Design, Furniture
Thanks go to: Lalya Gaye for continuous work evaluation and advice, Olof for being there whenever we needed help and to each other for being a great team in this successful collaboration. The whole is greater than the sum of the parts!

interactive table

Contents
Introduction Related Work Microsoft Surface. ReacTable. Other Projects. Hardware Building a frame. The webcam. Infrared Illumination. Projection. Software ReacTIVision. Processing. Evaluation User Study. Future Enhancements. Conclusion Literature Appendix 14

Introduction

The Aim of the project is to make a table, which can be used, in a home environment with an integrated multi-touch screen; with which people and objects interact. The table is designed to be customizable to the interactors and objects for interaction thereby providing various functions. The table is adaptive to the surroundings that means it acts according to the place it is situated. The idea of interactive multi-touch table comes from the concept of home automation and single computer for a home like Microsoft Home Server. It is the concept of providing interactive experience to the people with the devices and things they normally use in their daily life. Usually people use the objects but do not interact with them, our idea is to make object like tables more useful. For that we have taken a table and tried to change the upper surface of it into an interactive screen. The table reacts in a pre-programmed way to the objects placed on it. Also the table is adaptive to the environment, It can change itself according to the environment it is situated in. The core concept of the technology revolves around reacTIVision, which is a computer vision framework for table- based tangible interaction.

Related Work

Multi-touch interfaces have been around since a couple of years, but the history goes back to 25 years, when in 1982 University of Toronto developed a multi-touch display. Bell Labs also engineered a multi-touch screen in 1984. In 1999, a Newark-based company called Finger Works introduced a range of multi-touch products. Apple Inc. acquired the company in 2005 and used this technology in their 2007 launched iPhone, which is a landmark in using multitouch technology on a small scale, i.e. a phone. In 2001 Microsoft also started working on multi-touch displays, resulting in the launch of Microsoft Surface1 in 2007. Many other products have been introduced in the recent years, including products based on Tables, Walls, computer screens, and other innovative types of displays. Although being used at very small-scale like the iPhone, the technologys major use lies in the larger displays like tangible user interfaces, such as tables and walls, where more than one user can share a single display and interact with it in multitouch fashion.
Microsoft surface, December 2008. URL
http://www.microsoft.com/surface/.
Figure 1: The nal table prototype at the exhibition.
Microsoft Surface A number of professional products have been introduced in the market by companies such as Microsoft, that launched Microsoft Surface2 on April 17, 2008. This is a PC with windows Vista turned into a table, with multi-touch user interface, to manipulate digital content with hand gestures or physical objects. It is based on the same principles as ours: a projector is placed inside the table, which projects the computer screen on the reective surface in a acrylic frame. An infrared camera is used to detect nger blobs or physical objects, which touch the acrylic. Users interact with the table by touching or dragging their ngertips, or by placing objects with special tags attached to them. The camera detects the objects by detecting and recognizing these tags.

ReacTable ReacTable3 is another multi-touch table developed by the Music Technology Group, at the Universitat Pompeu Fabra, in Barcelona Spain4. This table is developed as a digital music instrument designed to help musicians and players by presenting an aesthetic tabletop user interface. Interaction with the table is achieved by placing blocks called tangibles on the tabletop, and by ngertips, creating music and sound effects.
reactable, December 2008a. URL http:

//mtg.upf.es/reactable/.

Sergi Jord, Martin Kaltenbrunner, Gnter Geiger, and Ross Bencina. The reactable*. In Proceedings of the International Computer Music Conference (ICMC 2005), Barcelona, Spain, 2005.
ReacTable uses the same functionality as the Microsoft Surface: a projector to project the computer screen onto the tabletop and a video camera to detect infrared light reected by the objects or ngertips. Here the objects are recognized by ducials, which are specially designed visual markups or tags attached underside the objects. These ducials allow to determine the position, rotation and movements of objects in any direction. The group has also developed a software library called reacTIVision to help process the multi-touch inputs such as blobs created by ngertips and objects.
Other Projects Many other similar works have emerged in the recent years, and the technology is widespread among companies, organizations, universities and independent hobbyists, creating a diverse set of products based on multi-touch table interfaces. NUI Group 5 is an Internet discussion group dedicated to the development of multi-touch tangible user interfaces. Many hobbyists have created their own multi-touch tables, with different techniques and materials. New companies have emerged in the eld, selling their own range of products. Instead of such widespread development in this technology, this product is only used at commercial places and for public use. Due to higher costs, the multi-touch table has not been able to enter consumers homes, where it can be best utilized. Many people consider the interaction technique used in the tangible user interfaces as natural, but this natural interaction has not reached a common computer user.
Nuigroup, December 2008. URL http:

//www.nuigroup.org.

Hardware
Basically the interactive table consists of a projector to project the computer screen onto the semi-transparent tabletop and a video camera to detect infrared light reected by the objects or ngertips.
Building a frame Designing the tables frame was one of the important parts of the project. Due to budget and time restriction we did not build the table from scratch, instead we modied two IKEA shelves to serve our purpose. The rst design of the frame was a nice one, but was a bit tall and the look was not aesthetic as well. We then arranged the two bookshelves to make a longer table, half of which was intended to be used

Figure 2: Painting the table.
as a screen, while the rest was aimed to provide longer distance for image projection. But this design looked odd as well, as only half of the table could be used as a screen while the rest is still a noninteractive space. As the nal design, after an inspiration by the newly built architecturally famous Kranhaus towers in Kln, Germany, by one of the group members we designed and built the table a little higher at one side (inverted "L") making place for the projector and also the noninteractive space became much smaller. Under the non-interactive space lies the projector, the camera, and the IR-LEDs used for illumination. Above the projector we placed a small mirror, to divert the image to the main mirror placed beneath the acrylic. The table is not destined for any particular usage such as a coffee table or a dining table. It has been designed to be general, making it useful as a coffee table, or a small dining table for two persons. Atop the table is the reecting acrylic to produce the image projected from the projector inside the table. In the process the rst constraint is the height. To produce the exact image which ts the screen (surface of the table), the distance should be large between the screen and the projector and that will increase the height of the table. To overcome this problem we arranged two mirrors at 45 degrees to the surface on opposite sides. This way, we enlarged the distance between the projector and the acrylic, which resulted in an increased and suitable image size. The IR-LEDs were mounted at the lower bottom of the table in such a way that the acrylic is illuminated as equal as possible. All other components like the Arduino microcontroller and the remaining circuit components were attached to the side of the table to not interfere with the projection beam, IR illumination and webcam view. The webcam For this project we used a Creative Live! Cam Optia webcam. The camera has a recording speed of 30 frames for second that met our requirement of 24 frames for second and gave us better result. The rotation angle of 270 degrees was also one of the factors, which made us go for it. Because of the integrated IR lter, webcams only operate in the human visual range, although the CMOS or CCD chip itself is also sensitive to the IR spectrum. Therefore we had to remove the IR lter so that it recognizes the IR light and serves its purpose. We carefully removed the IR lter from the webcam and used some developed but unexposed photo positives as an IR pass lter. As result we got a webcam, which is only sensitive to the IR spectrum. To not interfere with the projected image, the detection of
Figure 3: Kranhaus, Rheinauhafen Kln
Figure 4: Creative Live! Optia webcam
nger blobs and objects is based on the reected infrared light from the display, so the web cam needed to be modied in order to detect the reected rays of IR and lter out the visual spectrum.
Figure 5: A small piece of mirror is placed above the projector, which reects the projected image onto another larger mirror placed on the opposite wall. This larger mirror projects the image onto the acrylic.

Infrared Illumination Basically we wanted to control the intensity of the infrared light ooding the table because too much or too little intensity may distract the whole process. For that, we built various infrared light arrays which are seperately adjustable in intensity and direction. In addition we have deployed two fans from the backside to keep the temperature inside the table as optimal as possible. The IR driving circuits were controlled using the 6 PWM outputs of an Arduino Duemilanove microcontroller driving six TIP-121 ICs. In total, we used 6 IR-LED arrays, each consisting of 16 OSRAM SFH 485-2 LEDs. So, totally 96 LEDs were used to achieve a diffuse IR illumination of the acrylics backside. Projection The table has a low height, to meet its purpose of being used in a home environment like a coffee table. Unfortunately we could not
Figure 6: OSRAM SFH485-2 880nm Infrared LED
nd a short-range projector that would have otherwise met the goal easily when placed under the acrylic surface. So instead we used an ordinary long-range projector. This one alone did not produce the required size of image on the table due to a short distance between the projector and the screen. Therefore, after application of some physics we understood that we can achieve the goal with two mirrors placed at 45 degrees while the smaller faces the ground the larger faces up to the surface. This increased the projection distance and we got a larger image size that tted the dimensions of the table after some ne tunings. Finally we used an LG DX 130-JD projector after a couple of trials with other ones, which were available to us. This one has much higher lamp brightness but still being a long range one. As described in the "Projection" section we got rid off this problem by deploying mirrors to hit the targeted dimension for interaction.
Figure 7: Schematics for the ducial and nger tracking setup.

Software

The acrylic on top of the table serves as an interaction surface for the user. The reacTIVision6 (described in the tools section) software which processes videos in real time, acts as the intermediator between the users input and output. The inputs in our case are ducialmarked coffee cup, remote controller, music cards and of course nger blobs.
Ross Bencina, Martin Kaltenbrunner, and Sergi Jord. Improved topological ducial tracking in the reactivision system. In Proceedings of the IEEE International Workshop on Projector-Camera Systems (Procams 2005), San Diego, USA, 2005.
ReacTIVision The webcams videostream is processed by the reacTIVision software in realtime and the unique IDs, position and rotation data of the detected ducial marks and nger blobs are immediately send to the Processing application over the network connection using the established tangible user interface protocol TUIO7 which is basically a special implementation of the OpenSoundSource8 interface. ReacTIVision9 is an open source, cross-platform computer vision framework for the fast and robust tracking of ducial markers attached onto physical objects, as well as for multi-touch nger tracking. It was mainly developed as the tool kit for rapid development of table based tangible use interfaces(TUI) and multi-touch interactive surfaces by Martin Kaltendrunner10 and Ross Bencina11 at the music technology group at the Universitat Pompeu Fabra in Barcelona, Spain.

Martin Kaltenbrunner, Till Bovermann, Ross Bencina, and Enrico Costanza. Tuio - a protocol for table based tangible user interfaces. In Proceedings of the 6th International Workshop on Gesture in Human-Computer Interaction and Simulation (GW 2005), Vannes, France, 2005.
Opensoundcontrol, December 2008. URL
http://opensoundcontrol.org/.
reactivision, December 2008b. URL
http://mtg.upf.es/reactable/?software.
Martin kaltenbrunner, December 2008. URL http://www.iua.upf.es/~mkalten/.
Processing Then main application renders the visual feedback corresponding to the objects ID, position and rotation data, which is then projected on the acrylic. For example a "TODO" lists will be shown next to a coffee cup or the current television programs next to a remote control when it is placed on the table. To further boost the interaction we also added a future that enables the user to play music of his taste from music cards tagged with ducial markers.
Ross bencina, December 2008. URL
http://www.audiomulch.com/~rossb/.
Figure 8: Coffeecup with attached ducial.
The main application is written in Processing12 , which is an open source programming language based on Java, widely used for projects in interaction design. It is widely used today by students, researchers and artists. It was initiated by Casey Reas and Benjamin Fry, part of Aesthetics and Computation group at the MIT Media Lab. The language builds on the graphical capabilities of the Java programming language, simplifying features and creating a few new ones [Processing (Programming Language)]. It is licensed under GNU General Public License, and is available for different platforms, such as Windows, Mac and Linux. To enhance especially the graphical performance of the JAVA application, additional libraries were used. By using the GSVideo and GLGraphics libraries we were nally able to playback high resolution videoles in fullscreen. Furthermore the SQLibrary was used to establish MySQL connections to the database storing the different ducial IDs and properties.
Processing, December 2008. URL http:

//www.processing.org.

Evaluation
Generally speaking we are extremely pleased with the results obtained in such a short period of time since the main goal of the project was to have a working table at the end of the course. We wanted to make further interaction with the table but building the hardware took the lions share of the time. We strongly believe that the basic foundation for further work is already laid and our goal is almost achieved successfully bearing in mind further improvements can be made to come up with more interactions with the table.

User Study We wanted to do a user study in an actual home to test out the table in its intended environment, but the time was limited so this was not manageable. Instead the closest to a user study we came was the exhibition when a lot of people were asking how it worked and interacted with all the different objects we had for display. These feedbacks from the users at the exhibition were quite motivating that similar results can be extrapolated for the home.
Future Enhancements An interactive table can be a much valuable item in homes. It can serve as a game machine, a communication device, an automation pad, or just a funny thing to play with. Future works include connecting the table with the home server rather than using a dedicated
PC for it. More simple and useful applications can be designed for interaction with family members. Multi-touch games can be used for children to play on the table. At the moment nger interaction is not yet fully implemented, except the opportunity to draw some lines on the surface. Here we would like to add much more features such as clicking on the objects to see more info, drag the objects around, change their size and rotate them.
Figure 9: The graphical interface

Conclusion

When looking back on working with the project we can say that it took a lot more time than we rst thought. The main problem we had, was the distance between the projector and the surface, which was a vital part when coming up with design decisions for the table layout. As mentioned in the design process part, we did not get our hands on a short-range projector and this got us to redesign the table a few times, but now that it is nished we are really pleased with the result and it looks really nice. We also planned to come up with other applications but as discussed on the evaluation section, we believe that we were able to show some simple but interesting applications that one can deploy on the table. Besides these, we have had lots of fun while working together and the table was indeed a perfect christmas gift for each one of us from longer hours of work and a sweating but joyful ride.

References

The interactive table project website, December 2008. URL http: //web.student.chalmers.se/groups/uc08-7/. Martin kaltenbrunner, December 2008. URL http://www.iua.upf.es/ ~mkalten/. Microsoft surface, December 2008. URL http://www.microsoft.com/ surface/. Nuigroup, December 2008. URL http://www.nuigroup.org. Opensoundcontrol, December 2008. URL http://opensoundcontrol. org/. Processing, December 2008. URL http://www.processing.org. reactable, December 2008a. URL http://mtg.upf.es/reactable/. reactivision, December 2008b. URL http://mtg.upf.es/reactable/ ?software. Ross bencina, December 2008. URL http://www.audiomulch.com/ ~rossb/. Ross Bencina, Martin Kaltenbrunner, and Sergi Jord. Improved topological ducial tracking in the reactivision system. In Proceedings of the IEEE International Workshop on Projector-Camera Systems (Procams 2005), San Diego, USA, 2005. Sergi Jord, Martin Kaltenbrunner, Gnter Geiger, and Ross Bencina. The reactable*. In Proceedings of the International Computer Music Conference (ICMC 2005), Barcelona, Spain, 2005. Martin Kaltenbrunner, Till Bovermann, Ross Bencina, and Enrico Costanza. Tuio - a protocol for table based tangible user interfaces. In Proceedings of the 6th International Workshop on Gesture in HumanComputer Interaction and Simulation (GW 2005), Vannes, France, 2005.

Appendix

The following tools were used to build the nal version of the Interactive Table: Hardware 96 OSRAM SFH-485-2 IR LEDS (arranged in 6 arrays, 16 LEDs in each) 2 IKEA bookshelfs (LACK) modied to be a table frame Arduino Duemilanove 2 Mirrors (60x70 cm and 20x15 cm). LG DX 130-JD OLP Projector Creative Live! Cam Optia Macbook 2GHz, 2 GB Ram Unexposed, developed positive lm material as IR pass lter Wood blocks and strips Angle brackets of different sizes and shapes Various nuts and bolts Software reacTIVision 1.4pre2 GStreamer Processing 1.0.1 with TUIO Client, traer.Physics, SQLibrary, GSVideo and GLGraphics library