The overall theme of my research is human computer interaction (HCI). Within this area I have worked with theoretical, methodological and practical aspects of interaction design, development, and use of digital systems combining concepts from HCI, Art & Design, CSCW and Architecture.

Studying Domestic Interactions

Understanding people doing activities in their private settings, such as their home kitchen, and how they conduct cooperative and interpersonal activities with others, such as cooking together, can be difficult to study in-situ. There have been many different solutions to this problem. We propose a new way of accessing this information, called Digital Ethnography, specifically looking at YouTube videos to gain access to people’s homes and private moments and activities that they share with others. We adapted research and annotation methods from sociology for mapping peoples movements in a space to analyse the formations people make when they cook together. We then classified these formations with respect to different cooking activities that took place. This work provides a foundational understanding of domestic interactions, supporting the design of technological interventions for homes. This included work that focused on supporting intergenerational relationships over a distance, including through the act of cooking together.

Health, Well-being and Ageing

Health behavior change is a relatively new research area for technological intervention. Persuasive technology design and positive computing are both areas of interest for people looking into technology solutions for helping people to change behaviors that are detrimental to their health, or to increase people’s general well-being. We have worked with smokers to rethink the concept of the “mobile health app” analyzing what it can and cannot do to help people quit smoking. Through field studies with users, we discovered the idea of combining the convenience and just-in-place attributes of the mobile phone in the daily activities of a person who is trying to quit smoking, with the efficacy of personalized counseling. Taking advantage of the smartphones abilities to automatically register certain characteristics of context, coupled with simple user reporting on smoking events, we were able to create an app that reportedly helped people in their attempt to stop smoking. I have also been involved in presenting work with technology supporting people with dementia. Using Internet of Things technologies, including sensors embedded in everyday objects and identifying tags carried by residents, a system was designed to deliver relevant snippets of information to a carer’s smart watch. The information linked a resident to a nearby object that was meaningful to them, making it easy for carers to start a relevant and meaningful conversation involving the patient’s memories. We have also studied the elderly and creativity, doing field studies with MakeyMakey toolkits in their homes, inviting groups of older adults to explore technology and propose novel ideas of possible future applications for their everyday lives.

Indexing to Contextual Cues

Smartphones and smartwatches have many additional capabilities beyond calling others and finding out what time it is. They are now much more context aware. They are now able to locate people in space, know something about people’s movements, know about proximate other people with similar devices, or respond to sensors in the environment. They can be used to communicate with others through text and audio channels and they know what time it is and where they are. The problem that these devices have in terms of interface design is limited screen space. That is why it is either important to design the information communication between human and machine very carefully in terms of what is represented on the screens, which can be done using indexicality, or, to use alternative input and output modes such as gesture and speech. Indexicality is a concept borrowed from semiotics that allows designers to use contextual cues in the users surroundings to minimize the amount of information required on the screen by either implicitly or explicitly indexing to these spatial, temporal and social cues in the environment. Our work in this area focuses on the design and evaluation of indexical systems that leverage on their context to enrich communication.

Smart City

Augmented reality makes it possible to superimpose a layer of information over the world. Using the camera facility provided by smartphones we have made early explorations into how to augment the world with either facts or fictional content. Using the phone screen as the view-finder we have introduced the use of AR into several different contexts of use. Examples include, using AR on a mobile phone for knowing more about the city around you, for weaving fictional stories around familiar places that you travel to, for siting your new home on your block of land and creating redesigns in a contextual architecture mode, and for putting a layer of transport information on the world, such as how far away the next bus is from your current location.

Blended Spaces for Communication

Being able to work with others in a face-to-face situation allows people to negotiate and relate to each other in ways that are beyond spoken words, voice inflections and facial expressions. Adding shared documents into this workspace allow people to point to and move shared objects and information while working remotely with others. Our blended spaces project created a virtual shared space where people can work with documents over a video conference system, while at the same time being able to interact and relate to each other using natural modes such as eye contact and gestures, that one would use at a similar meeting held in the same physical space. Through careful design of physical space and interactive software objects, we created a unique environment where people who were remotely located could feel like they were talking and passing around documents as if they were co-located.