{"id":224482,"date":"2024-09-19T18:43:31","date_gmt":"2024-09-19T10:43:31","guid":{"rendered":"https:\/\/www.grab.com\/sg\/?post_type=editorial&p=224482"},"modified":"2025-02-26T18:56:38","modified_gmt":"2025-02-26T10:56:38","slug":"grabchat-translations-accurate-engineering","status":"publish","type":"editorial","link":"https:\/\/www.grab.com\/sg\/inside-grab\/stories\/grabchat-translations-accurate-engineering\/","title":{"rendered":"We made translations on GrabChat more accurate. Here\u2019s how."},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

Messages you exchange with your driver-partner in the Grab app are automatically translated. This GrabChat feature is meant to ensure that you\u2019re able to communicate with driver- and delivery-partners seamlessly, even when you don\u2019t speak the same language.\u00a0<\/span><\/p>

When we received feedback that these translations were occasionally inaccurate, we put together an engineering squad to tackle the issue.\u00a0<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
An example of a bad translation. The correct translation is: ok sir.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

The goal was simple. We had to improve the quality of translations on our platform, while keeping cost efficiency in mind.<\/span><\/p>

Building an in-house translation model<\/b><\/h5>

Initially, we completely relied on an external service to translate our chat messages. Although the third-party tool was generally effective, translations weren\u2019t always accurate.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
An example of an inaccurate translation from the third-party service recorded on Dec 19, 2023.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

We realised we could create our own translation system and even aim to outperform off-the-shelf tools that often lacked context. Our advantage? Our wealth of data which is directly related to the context of booking rides and ordering food\u2014and thus can generate more precise translations.\u00a0<\/p>

What are users saying?<\/h5>

For example, we found that many of the chats centred around pickup points. The exchange of courtesies such as \u2018Hi\u2019 and \u2018Thank you\u2019 were also common. Users also frequently used GrabChat to inform each other about their arrivals.\u00a0<\/p>

This allowed us to build a dataset that is representative of the types of chat messages exchanged between users and driver-partners, and diverse enough to capture all of the nuances of the conversations.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
Some examples of recurring topics include arrival notifications and pickup instructions.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

By aggregating and analysing past conversations, we can pinpoint recurring themes and topics. This valuable information can then be used to enhance the training of our translation models and improve their accuracy and effectiveness.<\/p>

Setting the quality bar<\/h5>

We then worked with Grab’s localisation team to set a standard for translations. The goal wasn’t to create a dataset large enough to fully train our model, but to get enough data that would help us set some benchmarks.\u00a0<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t
Good vs bad translations<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

These translations will also serve as a guide for the model to accurately capture the desired style and tone.<\/p>

How to train a model?\u00a0<\/h5>

To create our model, we also needed training data. We used an open-source Large Language Model (LLM) to create artificial translation data. The model had to be large enough to produce high-quality results and could handle the many Southeast Asian languages across the markets we serve.\u00a0<\/p>

This was especially important for languages like Vietnamese and Thai that consist of large character sets and diacritics\u2014marks, shapes or strokes that accompany letters.\u00a0<\/p>

(Read more: Why we redesigned a typeface for Thai and Cambodian scripts<\/a>)<\/strong><\/p>

We then used some of the translations with high benchmark scores as data for our model to learn from. After all, the quality of translations by Language Learning Models (LLMs) is only as good as the data they are trained on.<\/p>

Fine-tuning our model<\/h5>

Before our translation system was good to go, we also had to make sure that elements such as numbers or unique symbols were not misinterpreted or omitted during translation. This is critical as displaying incorrect numbers in a translation can confuse users.<\/p>

The model pulls out all non-translatable items from the original message, tally each occurrence, and then attempt to find a corresponding match in the translation. If the match is not found, we reject the internally generated translation and revert to using an external third-party translation service.<\/p>

Expanding our solutions\u00a0<\/h5>

We believe that our proprietary in-house translation models are not only more cost-effective but cater more accurately to our unique use cases compared to third-party services. We will focus on expanding these models to more languages and countries across our operating regions.<\/p>

We are also exploring opportunities to apply learnings of our chat translations to other Grab content. This strategy aims to guarantee a seamless language experience for our rapidly expanding user base, especially travellers.<\/p>

The problem of language translation and translation quality benchmarking is highly complex. If you\u2019d like to know more, for example about language detection (how do we know which languages which languages to translate from, and into?) and the measures required to do all of this at manageable costs, dig into\u00a0this post on Grab\u2019s engineering blog<\/a>, which gets into these points with more detail.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"parent":180237,"menu_order":0,"template":"grab21-default","acf":[],"_links":{"self":[{"href":"https:\/\/www.grab.com\/sg\/wp-json\/wp\/v2\/editorial\/224482"}],"collection":[{"href":"https:\/\/www.grab.com\/sg\/wp-json\/wp\/v2\/editorial"}],"about":[{"href":"https:\/\/www.grab.com\/sg\/wp-json\/wp\/v2\/types\/editorial"}],"version-history":[{"count":25,"href":"https:\/\/www.grab.com\/sg\/wp-json\/wp\/v2\/editorial\/224482\/revisions"}],"predecessor-version":[{"id":224527,"href":"https:\/\/www.grab.com\/sg\/wp-json\/wp\/v2\/editorial\/224482\/revisions\/224527"}],"up":[{"embeddable":true,"href":"https:\/\/www.grab.com\/sg\/wp-json\/wp\/v2\/editorial\/180237"}],"wp:attachment":[{"href":"https:\/\/www.grab.com\/sg\/wp-json\/wp\/v2\/media?parent=224482"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}