— About this Project —
This project is an exploration of tools to improve communication between children with and without disabilities in inclusive education. The designs focus on supporting social communication, highlighting each child’s authentic voice, and honoring and extending each child’s existing communication behaviors.
— Background —
This project was my undergraduate thesis at RISD. The ideas here have been the foundation of all of my work related to assistive technology ever since. Click here to see a complete book about this project.
When children with social and communication related disabilities are included in classrooms with typically developing peers there can be extra challenges to building social relationships and community. The goal of inclusion is to teach kids to value each other regardless of race, academic ability, physical ability, appearance, gender, etc. The best way to do that is for everyone to be able to share their story, but the resources and technology available do not always support children who communicate without words’ ability to participate in this way. This project started with these questions:
- How can I redesign AAC (Alternative Augmentative Communication, the technology that supports folks with communication related disabilities) to ensure every child has a voice to tell their story?
- How can empowering everyone to communicate more fully improve the experience of inclusive education for everyone involved, including students with disabilities and typically developing students?
These are examples of some of the students I worked with:
Daniel is in second grade. He doesn’t speak, but uses some sign language words and answers yes and no questions. He is extremely friendly and motivated by being with his friends. Sometimes his disruptive behavior can get in the way of class.
Elizabeth is in third grade. She is very intelligent and thoughtful, but her speech is difficult to understand. Often people mistakenly assume she doesn’t understand them, because they find it difficult to understand her.
James is in second grade. He doesn’t speak at all and it can be difficult to tell how much he understands. Often he resorts to aggressive means to get attention and express himself, such as screaming or hitting. It is important for him to be with his classmates because he is much happier and more energized when he is with friends.
— Design —
The system I designed was based on two very important values that emerged from my research:
- Language is social. The inclusive communication device for the classroom must emphasize communicating personality and emotion, as a priority over needs based and academic communication. This is important because all too often, AAC systems emphasize requesting and academic content ahead of social interaction.
- All behavior is communication. The inclusive communication device will make use of and bring attention to the existing communication abilities of each child. This is important because it can be easy to misunderstand or dismiss the nonverbal communication behavior kids engage in as “just behavioral” instead of examining it’s meaning.
The criteria for a successful design I developed are the following:
- The system will utilize and reinforce the child’s existing means of communication.
- The system will use emotional communication that is not linguistic to increase the range of expression and add authenticity.
- The system will empower the child to express their personality and social identity
The system I designed has three parts:
- Gesture sensors which respond to a child’s unique communication behavior
- Badges which are activated by the gesture sensors and draw attention to the meaning of the child’s communication behavior, as well as resources and tools for continuing a conversation.
- A “social atlas” which provides information about how the child communicates.
This system allows the child to use their existing gestural communication to express them self to their peers and caregivers. It incorporates gesture sensing, an immediate expressive utterance based on the child’s personality and the emotion of their gesture to get attention, and an introductory”guide” to the child that explains how best to communicate with them using their available means.
The child communicates using an existing gesture they already use to communicate. When they get the system there is a training period when their caregiver presses a button on the sensor they’ve chosen every time the child does the gesture, so the system can learn the signature. Later, whenever the child does that action the system will be activated.
The child and caregiver choose an icon for the child’s badge to convey who they are. Each badge represents a personality type, and is capable of a full range of emotional expressions related to that personality. It is important for a child who is normally bubbly and fun to be able to make an angry expression, and vice versa.
The first set of badges were symbolic of characters, this is an alternate set that are more abstract. They represent emotions and valances but not in case some users who feel uncomfortable representing themselves with a “spokesperson” or avatar.
The social atlas is a social and communication guide to child. It also has a badge on it to draw attention to the full range of communicative abilities the child has. This way if someone meets them they will have an opportunity to communicate with them without frustration. The atlas is plush and portable and low tech so the child can always keep it with them, even if their other devices are unavailable. It has audio buttons for listeners who need audio support and contains extra pockets for additional helpful materials such as symbol boards or sign language guides.
The cards in the social atlas emphasize the ways the user and listener can contribute to a positive interaction. You can see how this ideas has evolved here.
These prototypes were built with Arduino to experiment with how the device would be received in a classroom setting. I based the sound effects on characters from WALL-E and Stars Wars, examples of excellent nonverbal communicators.
During user testing in this phase, I was interested in how the expressive badge would influence the classroom setting. For the child this device is unlike the “real” experience of the device because it is not gesture activated. This prototype provided the opportunity to observe the child’s peers and caregivers’ reactions to the child’s new voice. You can also see it on a classmate during my final review.
— Process —
To research the project I first became familiar with existing communication devices.
I met with a graduate student in Performance Studies at Brown, who introduced me to Subaltern studies. She brought up concerns about giving voice to people and whether it would be empowering and authentic.
After our conversations I decided to include a non-linguistic means of communication and looked to characters from media who communicate effectively without language.
In order to consider what messages it might be important to share, I asked my classmates about their communication. These cards are their answers to the question: “What is one thing about your personality you wish other people knew, but it isn’t immediately obvious?”
Blendie, By Kelly Dobson, MIT Media Lab. Blendie is an exploration of ways people can interact with machines using behavior instead of buttons. The blender is turned on by growling at it to match the sound you want it to make.
Nintendo Wii is a way of interacting with technology using gestures and behaviors, not keys and buttons.
Social Mobiles, Ideo. These mobile phones explore ways of communication that go beyond speaking into the phone. The designers were interested in changing the way people behave with their phones in public spaces.
The Tango by Richard Ellison and BlinkTwice is the first communication device that uses tone of voice to give users control over how they sound to others.
Kiss Communicator, IDEO. Kiss Communicator is for couples who are at a distance. It is activated when one partner blows into it and a pattern appears on the other’s device that is only visible for a moment, like a kiss.
Biosensors recognize patterns in the body such as heart rate and blood pressure and they can be used to provide clues to emotional conditions.
Somiya Says was designed by Joanna VanDaalen and Somiya Shabban to empower Somiya to share her personality, because people often assume she isn’t intelligent because her speech is unclear. Graham Pullin writes “The short term utterance is also a long term badge— a label of Somiya’s own devising, to express her individuality and identity, rather than any stereotype associated with her impairment”