Grab’s AI Voice Assistant lets visually impaired users book rides with ease

Powered by AI and LLMs

AI Voice Assistant uses OpenAI’s GPT-4.1 Large Language Model and voice recognition to interact with users through voice commands. It guides users through the process of booking a ride and keeps users updated on the driver’s status, from the moment a driver is found to their arrival. Other capabilities include: 

• Predict passenger’s drop-off & pick-off locations, while allowing them to make changes 
• Suggest ride options to passengers
• Provide post-booking status updates. 
• Allow users to cancel the ride or call drivers 
• Change payment methods, for example from cash to cashless

Building for local contexts

Creating Voice Assistant for the visually impaired requires our AI models to grasp the context and subtleties. For instance, we noticed the model struggled to recognise Singaporean accents accurately.

Built on OpenAI, we fine-tuned the speech-to-text model by training it with 80,000 local voice samples. These were contributed by Singapore-based Grabbers who recited building names, landmarks, and other phrases to improve the model’s understanding of local accents

This improved the recognition accuracy from 46 per cent to 89 per cent!

You can help us train the model

We want to further improve its accuracy, which is why we’re launching a voice donation campaign in June to invite Singapore users to contribute more voice samples.

Apart from fine-tuning our models, we had to put the feature to the test before rolling it out. Over the past few months, we have been conducting user testing among visually impaired users, with the help of SAVH. This allows us to understand their unique needs, challenges and ensure that our tool is intuitive and easy to use. 

Voice Assistant is currently being piloted in Singapore and is available to all users who have the talkback feature enabled on their phones. 

We believe that Voice Assistant is a significant step towards a more inclusive app. We will continue expanding its use cases to help more people carry out transactions, connect with loved ones, and access our services with ease.