Homework 3

AR for text-translation

AR for realtime text translation maybe one of the first serious killer apps for the technology. Advancements in machine learning, computer vision and natural language processing technologies mean that we are getting better at text translation every year. It may not seem like a must-have functionality in a largely monolinguistic country like the USA, where most of the text we read is in English. Even when a foreign language is used here, it is because the text is intended at readers who know the language. Outside of tourism, there are not many instances where people here fill disadvantaged because of the language they grew up steaking. That’s not the case with several places in the world, like say Europe or India where the linguistic diversity of a community is not reflected in the medium of communication used in pubic forums.

For example, In the city in which I grew up, Bangalore, the percentage of migrant population from other states is very high. The city is a mix of people from different regions of India, which have their own languages, dialects and scripts. The migrant population often doesn’t have enough time to learn the local language and the script and are often left confused when they see important information such as city bus destinations in a language they don’t understand. The problem is solved to some extent by making sure every instruction and information is represented in the local language (Kannada) and English. This solution assumes that the migrants are well versed in English, which is definitely not true especially for migrants who are from villages where English is almost never a medium of communication.

In such instances, real-time, affordable AR-based text translation would bring the migrants to an equal footing with the local population and make their lives easier. This would also mean that more people would migrate from one place to another and still be able to maintain their cultural identity while being able to easily navigate through their daily life. The same problem and solution are also applicable to European countries.

Today’s most common medium for text translation in the real world is our phones. While phones may be able to solve translation problems one text at a time, the truly ambitious end goal of the technology is affordable AR glasses (or contact lenses) which translate and superimpose text in real-time. When we reach this stage where AR glasses are as ubiquitous as mobile phones today, we will have to rethink how we present information in the real world. Why not print a QR code on a board sign instead of text if all viewers will be seeing the board through their glasses? The information and the text could be personalised based on the user. True AR will have several socioeconomic consequences that we might not be able to predict at this point.

It is when we get to this level of technology that we will start asking questions about what unintended consequences we can cause. We could argue that this layering of synthetic data over real text might be harmful if the translator has malicious intent. If misused, the user might be given misleading information and the translator can create a “fake” reality for the user. Is personalised content always a good thing? Is it safe to hand over the responsibility of translation to a third party? Who would be held responsible for any unfortunate incidents due to the failure of the technology? While we need to be cognizant of the repercussions, they shouldn’t deter the progress of a technology which has huge benefits to our society. We need to treat these problems as a challenge to overcome and build effective systems to prevent such undesirable outcomes.

Real-time translation testing

Since it is hard to find foreign text in my neighbourhood, I decided to test how well English translates into an Indian language Hindi, and have included one instance of conversion attempt into English.

1. Campus map.:- The original text reads “West side Campus Map”.
   
   While it has managed to change the script from English to Hindi, the context has not been fully converted into this language. “West” should have been translated into “पश्चिम”, and the Hindi equivalent of “Campus” is not used either.
   
   
2. Road Sign:- Parking for schools from 7 AM to 5 PM, only on MON-FRI
   
   This sign has been cleanly converted, but the time, the word “TOW” and the minimised representation of the days of the week have been left as is.
   
   
3. Speed limit sign:- Speed limit 20, on school days when children are present.
   
   Everything except the speed limit 20 has been translated well, which is curious because the translator has definitely picked up the text “20” from the board.
   
   
4. Newspaper clipping in a foreign language:-
   
   Original vs translated photo

 

While the translator does a fairly decent job of detecting words in the clipping, the conversion of the zodiac sign into its equivalent English word is hit or miss. The translator is able to pick up some keywords from the passages but the translation is very broken and incoherent

Concluding, we have a long way to go before a user who does not know Hindi can fully rely on the technology in their daily life. One more thing that caught my attention is that the same texts which failed to translate live , translated fairly well when run on google translate independently. This could mean that while we have gotten better at recognising individual words, the software still struggles to find context between the words if they are ordered in a non-standard fashion.