Designing Beyond the Five Senses

Think of the last time you used UPI to pay. You scan the QR code, enter the amount, enter your PIN or use biometrics, and you are done. You hear or feel the phone confirm the transaction at lightning speed. You might not even look at the screen, as you know it has already occurred. It is a simple example of design thinking in practice. The phone is engaging multiple human senses to communicate information. Design is going way beyond the visual.

Imagine going way beyond this. What if your phone told you the direction of magnetic north? Imagine a device that allowed you to see heat. What if you could hear sounds that are beyond your range of hearing and feel them as vibrations? What if a blind person could get visual information on a different part of their body? These ideas are no longer futuristic. Technology is enabling us to enhance and substitute human senses. This has profound implications for designing human experiences. Designers grapple with what a human being should be able to sense and why.

We have not grown up with five senses

We have grown up learning about five senses such as seeing, hearing, touching, tasting and smelling. But our experience of the world is much richer. We know where our hand is without looking. We know what it means to lose balance. We can sense pain, hunger, fatigue, temperature and what is happening in our own bodies. This information is as much a part of experiencing the world as the five classical senses. Technology can enable us to enhance some of these capabilities.

We can get information that our own biology would not normally get. Infrared light is invisible to us but visible to a thermal camera. It could be processed and presented to us as a visible image. That is a simple but powerful thought. Technology is enabling us to create new relationships between information and human perception. That is when design gets interesting. There are different ways technology can enhance human senses. Sometimes it enhances our current capability. For instance, night vision and thermal imaging can enable us to see something that otherwise we would not be able to see. Sometimes it substitutes one sense with another.

Also Read: How Transdisciplinary Education Shapes the Future of Learning

The BrainPort uses a camera to capture images and uses electrical stimulation to send information to the tongue. The tongue does not literally see the images that the camera captures, but users can be trained to interpret the pattern as an image. Similarly, the vOICe converts images into sound. It is fascinating to realise that the human brain can be trained to interpret these signals as visual information. There is also the possibility of addition. We add information that humans would not have sensed. North Sense uses vibration to indicate magnetic north. Humans do not naturally sense magnetic north, but some animals can.

This makes us rethink the definition of an interface. An interface can be beyond the screen. It could be vibration, sound, temperature, motion or perhaps smell. There may even be interface capabilities that we have not discovered yet. Does this mean that humans need more information? Perhaps not. I will be cautious here.

Technology has made it really easy to add information. We can make the screen busier. We can add more notifications, information in our ears, haptics on our body, digital information around us using augmented reality and so on. But the capacity of the human body has not evolved at the same rate. Our attention, working memory and senses get tired. We take time to learn a new code. This is especially pertinent in the age of AI. Artificial Intelligence can generate infinite information about everything. It can summarise for us, alert us, recommend for us and predict for us. It can observe multiple streams of information simultaneously. But can a human hold all of this in their head? We sometimes confuse more information with better understanding. They are not the same.

Think about driving in a busy Indian city. There may be vehicles coming from all sides, pedestrians, honking vehicles, signals, road markings, two-wheelers and all sorts of unexpected movement. Adding another icon or alert on the windshield may not always be useful. A small vibration on the steering wheel may sometimes communicate the message better. The design decision should come from the situation. Not from the fact that the technology is available.

We need to design for perception and not just for the device

This will be one of the important challenges for experience designers in the future. We have spent years designing products and interfaces. Now we are getting into the fascinating space of perceptual design. Take Augmented Realty, for instance. It is tempting to think that the more information we can cram into AR, the more valuable it is. But think of the experience of a person wearing AR glasses in the crowded railway stations of Mumbai or Delhi. The signs, people, trains moving, announcements, ads, movement around the person are part of their visual field. Now imagine that we overlay another layer of digital information on top.

The key challenge is no longer to get information on to the screen. It is to decide what information to hide. What information should stand out? What information should be pushed to the periphery? What information can be hidden until the user needs it? An effective AR experience may be quiet. It may only indicate something when it is necessary. That is why concepts like progressive disclosure, contextual highlighting and visual hierarchy become important. The designer is not only deciding what to show but also what to withhold. We need to manage the human's attention.

The human body is also an interface

Another fascinating area is touch. There are parts of the body that are more sensitive than others. Our fingers can detect far more detail than our torso. There is scope for a lot of experimentation for designers. Vibration can indicate direction, help us learn a movement, and carry information while eyes and ears are busy with other tasks. We already see some simple examples in products today.

A phone vibrating in the pocket can indicate an alert. A smartwatch vibrating in a particular manner can indicate a direction. The future will bring more subtle and sophisticated haptic languages. But then there is another challenge. If a wearable device vibrates constantly, we may stop noticing it. The wearable becomes heavy or uncomfortable. A rich haptic language may take too long to learn. So, the challenge for the designer is not to make haptics more sophisticated just for the sake of it. The designer should create meaning through the medium. A good haptic experience should not be intrusive or attention-seeking. It should feel like a part of a larger experience.

Smell and taste are different

Some senses are far more difficult to digitise. Smell and taste are two such senses. We can control lighting and sound fairly easily. It is much more challenging to control chemical reactions in the air. Smell is also linked to memory. The smell of agarbatti in Diwali, the damp soil after the first monsoon rain, and the smell of certain foods being cooked at home are all deeply personal and cultural memories. They are far more complex than sound or visual information. A smell may transport you to a particular moment or feeling. The same smell can mean different things to different people. It can be comforting or downright unpleasant. Designing digital scent experiences is incredibly challenging.

Also Read: Everything Around You Is A Design, Which Has Been Tailored To Perfection

There are fascinating explorations in electronic noses, scent projectors and devices that can stimulate taste with thermal, chemical and electrical stimulation. But the design challenges are far more complex than the technology itself.

There is no one-size-fits-all human being

There is another interesting design challenge when we enter this space of designing beyond the five senses. For whom are we designing? We often design for an imaginary average user, but the average user is a myth. Humans differ in hearing, vision, touch and other senses. Age, experience, environment and other factors influence our senses. This is an important consideration for a country like India.

We design for people who speak different languages, have varying levels of digital literacy and live in diverse environments. A young professional in Bengaluru may have a very different experience from a farmer in rural Gujarat. Even the language in which information is delivered may need to be different. It is not always a question of more technology. Sometimes it is a question of the wrong human being. Sensory augmentation allows us to rethink this approach. Instead of one-size-fits-all, future systems may need to be sensitive to the individual. It can become an important part of inclusive design.

The future may not involve an interface

We have long associated the word interface with a screen. That is no longer going to be the case. An interface can take many forms. It can be vibration, sound, spatial location, movement, smell and a combination of various senses. The line between technology and perception may blur in the future. We may have neural interfaces, sensory substitution, AI that can interpret information and decide what is relevant for us. It opens up exciting opportunities, but there are also uncomfortable questions. If we can have new senses, does that mean that some people will have more information than others? Do we really want to be exposed to more information constantly? Should an interface always tell us what is happening around us? What happens when technology fails us? The most uncomfortable question of all is: are we just expanding human information or human experience? These are two very different things.

Designing what matters

I believe this is where the future of design gets exciting. The next generations of designers will have access to technologies that we can only dream of today. AI can detect patterns, sensors can detect what humans cannot detect, augmented reality can overlay information on the environment, and haptics can communicate with the body and so on. There may be new interfaces that do not rely on a screen. But having the capability does not mean that we should exercise it. Designers will have to make judgements.

Sometimes, the best thing to do is to enable a new sense. But sometimes it may be better to remove some information. Sometimes it may be important to make the invisible visible. Sometimes, however, the best thing to do may be to leave the human senses alone. For student designers, it means that just studying technology will not be enough. For industry, it means that innovation will not be measured by how much technology has been added to a design. For design educators, it means that we must prepare our students to understand technology, human perception, attention, culture and judgement.

Most importantly, we need to equip them with the ability to exercise judgement. Technology will continue to expand the frontiers of what is possible. The challenge will be to decide what is meaningful. We are moving from designing what people see and use to designing what people perceive, understand and feel. That, for me, is the essence of designing beyond the five senses. Because the future of design is not about giving humans more senses but about helping humans to sense what matters.

About the Author:

Bhaumik Nagar is the Principal Faculty member at the National Institute of Design, Gandhinagar. With more than 30 years of experience across technology, design and education, he has worked at the intersection of emerging technologies and human-centred design. He has also served as Director of NID Andhra Pradesh and has contributed across different departments at NID.