Comment on page
Listening to intents over MQTT using JavaScript
In this tutorial, you will learn how to create a standalone JavaScript application that listens to various events coming from the Snips Platform, in the form of MQTT messages.
The Snips Hermes Protocol is the set of messages and exchange of messages between the various components of the Snips Platform. The messages are MQTT messages. The figure below describes the flow of messages during a simple voice interaction session.
You will most likely be interested in the following messages:
hermes/hotword/<hotword_id>detected: when a wakeword is detected, this sent, and Snips will listening to your voice command. You can for instance use this message to play a little tone, or start a led animation, indicating the start of a listening sessionhermes/asr/textCaptured: when listening, Snips will transcribe your query into text, and after a small period of silence, it will stop listening, and send this message. You may use this to play another tone, or stop a led animation, indicating end of listeninghermes/intent/<intent_name>: after text has been captured, the NLU module will process the text and transform it into an intent, which is the final representation of your query that you can use as an actionable item. This is where you want to put your intent handler code
When launched, Snips starts an MQTT broker that it uses to broadcast the above MQTT messages. We can connect to this broker using the MQTT.js module:
$ npm install -g mqtt
The connection code looks as follows, replacing
MQTT_BROKER_HOSTNAME with your broker hostname or IP:var mqtt = require('mqtt');
var HOST = 'MQTT_BROKER_HOSTNAME';
var client = mqtt.connect('mqtt://' + HOST, { port: 1883 });
client.on('connect', function () {
console.log("Connected to " + HOST);
});
client.on('message', function (topic, message) {
// When receiving an MQTT message, trigger and action...
});
When a wakeword is detected, a message is sent on the
hermes/hotword/<hotword_id>/detected topic, where hotword_id is the ID of your wakeword. The default value is default, which we will use here. We extend the above example to detect a wakeword and trigger an action as follows:client.on('connect', function () {
console.log("Connected to " + HOST);
// Subscribe to the hotword detected topic
client.subscribe('hermes/hotword/default/detected');
});
client.on('message', function (topic, message) {
if topic == 'hermes/hotword/default/detected' {
console.log("Wakeword detected!");
}
});
We now extend our example to also trigger and action when an intent is detected by the NLU component. In this case, a message is sent on the
hermes/intent/INTENT_NAMEtopic, where INTENT_NAME is the name of our intent.client.on('connect', function () {
console.log("Connected to " + HOST);
// Subscribe to the hotword detected topic
client.subscribe('hermes/hotword/default/detected');
// Subscribe to intent topic
client.subscribe('hermes/intent/INTENT_NAME');
});
client.on('message', function (topic, message) {
if (topic == 'hermes/hotword/default/detected') {
console.log("Hotword detected!");
} else if (topic == 'hermes/intent/INTENT_NAME') {
console.log("Intent detected!");
}
});
We don’t have to manually subscribe to the topic for each intent. If we want to trigger an action when receiving any intent, we can simply subscribe to the wildcard topic
hermes/intent/#:client.on('connect', function () {
// Subscribe to any topic starting with 'hermes/intent/'
client.subscribe('hermes/intent/#');
});
In the following example, when receiving an intent, we parse the payload to extract the slots associated with our intent:
client.on('message', function (topic, message) {
if (topic.startsWith('hermes/intent/INTENT_NAME')) {
var payload = JSON.parse(message);
var name = payload["intent"]["intentName"];
var slots = payload["slots"];
console.log(`Intent ${name} detected with slots ` +
`{JSON.stringify(slots)}`);
}
});
This completes the basic tutorial on how to write an MQTT client in Javascript that triggers actions upon receiving wakewords and intents from the Snips platform.
Last modified 5yr ago