Week 8

Audio - visual aids and consequences

The advent of deep learning and other technology breakthroughs have allowed computers to become really good at sensing and recognising information from the real world. In some cases, computers can make use of sensors like infrared and wifi detection to see much more than what humans can. We are also getting better at auditory input recognition, and natural language processing. Computers are able to listen, understand and act on our voice commands. We are fast approaching the realms of what was science fiction a decade or 2 ago.

In a future where regular people have access to this technology on their eyewear, I will discuss a few possible outcomes of this technology. In this future, our eyegear would become our primary display for all information. The eyegear would have to allow user to choose the amount of information they want to receive and what streams of data they want to subscribe to, what level of priority should this information take. The main purpose of this technology will be to augment our senses to make our lives more comfortable and to enrich our view of the world.

We can use the visual capture to improve our vision. The display could show us the path as we walk down a road, to a parked car or to a friend who has shared their location. We could be shown the traffic signal and the time left when our view is obstructed by other vehicles. It could help us out when we forget a person’s name by popping it over their head. In a store, It can help us find that particular flavour of chips packet we have been looking for all over. It can help tell us which vegetables in a store are likely spoilt. It can help us translate languages when we are in a foreign country, and alert us of a dangerous situation. We may be able to use AI to recognise our pet’s moods, (and maybe to warn us to stay away from a stray animal)

The auditory capture can help us translate languages, and also help improve our hearing by removing noise around the speaker’s voice. We can have live subtitles next to a person to help us understand each other better. We will be able to replay announcements we weren’t paying attention to.

Humans are very good at identifying audio and visual cues, the real uncharted territory for our senses is being able to perceive information that we don’t typically have access to. Could we convert mobile signal-strength to a fog, to help us navigate to areas with better strength? Can we use infrared capture to warn us of hot objects? Can we visually represent sound in our display to show us where a sound is coming from?

I have gone over a few advantages that everyone would experience, there could be advantages for specific use-cases based on the user’s profession. It could help doctors diagnose based on a patient’s wound by providing information that the doctor might have missed. It could help provide measurements of a structure to a plumber and many such specific use cases. While the amount of information might seem high, we can build a non-intrusive UX around this medium, by giving the users complete control over what information to display in which context.  Also, we humans are very good at adapting to a new kind of information display and UX.

Ethical concerns

While there are good UX challenges to tackle in this future, data privacy is the biggest challenge to get right. An argument often heard against these technologies is the fear of data breaches or worse, willful malicious use of data. On device machine learning is one proposed solution to bypass this problem. I believe that if there are sensors on a device, heavily regulating where the information collected by the sensor propagates to is vital in making sure it doesn’t end up in the wrong hands. Lawmakers will have to move faster than the improvement of technology to setup guidelines and best practices and set up fail proof systems to prevent such scenarios. While our past record with handling new technologies haven’t always been positive, we can be hopeful that we get it right in the future as the advantages of using this technology is a huge boost to human comfort.