Voice assistants, such as Amazon Alexa and Google Assistant, are being used daily by hundreds of millions of people at home, in the car, or on the go. Even though speech is natural and effortless way of interacting with devices and services, voice control is not yet widely used in factory environment. As part of the Reboot IoT Factory challenge, Nokia and Creoir wanted to bring voice control into Nokia Conscious Factory to validate the voice user interface in a factory environment.
Creoir Oy is a technology and product development company specialized in speech technologies and voice-enabled products and services. Creoir Offline Voice Solution SDK (Software Development Kit) enables a flexible and effortless way of bringing Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) to devices that need to work in challenging noise conditions.
Hands-free at the Factory
The operators and workers at the factory have typically many concurrent tasks at hand. The challenge is that operators need to interrupt the current tasks and free their hands to access instructions and manufacturing data, or to control computers and machines.
Security and reliability are extremely important in any factory. Therefore, it is not possible to use major cloud-based voice platform providers like Amazon or Google. Also, to reach robust speech recognition performance, attention needs to be paid to noise reduction and voice processing.
Smart Factory Voice Assistant
The solution in the PoC project was to create a Smart Factory Voice Assistant, which enables a hands-free voice user interface to access Nokia factory data. Creoir Offline Voice Solution SDK was used to create custom wake-word and needed voice commands to access the Nokia factory system. Special voice processing and noise reduction algorithms were deployed to guarantee far-field voice control even from a distance of five (5) meters.
The Smart Factory Assistant is a stand-alone product with an integrated speaker and two microphones. All speech processing is done on-device, and MQTT (Message Queuing Telemetry Transport) interface is used to access the Nokia factory back-end.
The deployment in the project was straightforward; at the beginning of the project MQTT topics based on intents were agreed upon between Creoir and Nokia. Creoir was responsible for speech to meaning conversion and the Nokia team implemented the needed action code at the Nokia back-end.
The noise conditions were recorded on-site, and these recordings were used in the configuration and parametrization of the noise processing algorithms.
Benefits and Next Steps
Smart Factory Voice Assistant is now in use at the Nokia Conscious Factory control corner to provide an easy way of getting quality data of the ongoing production. The plan is to extend use cases to support operators during the assembly process on the factory floor. Also, voice control for mobile robots is being investigated.
There are also many other opportunities for voice control in a factory environment. Feel free to contact Creoir if you would like to learn more about adding voice to your factory!
“Speech is the most natural way of interacting between people. At Nokia, we wanted to find out how we could use voice control for machines in a factory environment. Working with Creoir was a good experience for us – their expertise in speech technologies and embedded products was essential to achieve the targets of the PoC project. We are looking forward to continue co-operation in the future” – Jouni Herronen, Technology Program Manager, Nokia.
This project was part of the Grand Challenge 4. Read more: GC 4: Digital Production
Find out more about Creoir Oy here.
