Listening to intents over MQTT using JavaScript

In this tutorial, you will learn how to create a standalone JavaScript application that listens to various events coming from the Snips Platform, in the form of MQTT messages.

The Hermes protocol

The Snips Hermes Protocol is the set of messages and exchange of messages between the various components of the Snips Platform. The messages are MQTT messages. The figure below describes the flow of messages during a simple voice interaction session.

You will most likely be interested in the following messages:

• hermes/hotword/<hotword_id>detected: when a wakeword is detected, this sent, and Snips will listening to your voice command. You can for instance use this message to play a little tone, or start a led animation, indicating the start of a listening session

• hermes/asr/textCaptured: when listening, Snips will transcribe your query into text, and after a small period of silence, it will stop listening, and send this message. You may use this to play another tone, or stop a led animation, indicating end of listening

• hermes/intent/<intent_name>: after text has been captured, the NLU module will process the text and transform it into an intent, which is the final representation of your query that you can use as an actionable item. This is where you want to put your intent handler code

Listening to MQTT events

When launched, Snips starts an MQTT broker that it uses to broadcast the above MQTT messages. We can connect to this broker using the MQTT.js module:

$ npm install -g mqtt

The connection code looks as follows, replacing MQTT_BROKER_HOSTNAME with your broker hostname or IP:

var mqtt = require('mqtt');
var HOST = 'MQTT_BROKER_HOSTNAME';
var client  = mqtt.connect('mqtt://' + HOST, { port: 1883 });

client.on('connect', function () {
    console.log("Connected to " + HOST);
});

client.on('message', function (topic, message) {
    // When receiving an MQTT message, trigger and action...
});

Detecting a wakeword

When a wakeword is detected, a message is sent on the hermes/hotword/<hotword_id>/detected topic, where hotword_id is the ID of your wakeword. The default value is default, which we will use here. We extend the above example to detect a wakeword and trigger an action as follows:

client.on('connect', function () {
    console.log("Connected to " + HOST);
    // Subscribe to the hotword detected topic
    client.subscribe('hermes/hotword/default/detected');
});

client.on('message', function (topic, message) {
    if topic == 'hermes/hotword/default/detected' {
        console.log("Wakeword detected!");
    }
});

Handling intents

We now extend our example to also trigger and action when an intent is detected by the NLU component. In this case, a message is sent on the hermes/intent/INTENT_NAMEtopic, where INTENT_NAME is the name of our intent.

client.on('connect', function () {
    console.log("Connected to " + HOST);
    // Subscribe to the hotword detected topic
    client.subscribe('hermes/hotword/default/detected');
    // Subscribe to intent topic
    client.subscribe('hermes/intent/INTENT_NAME');
});

client.on('message', function (topic, message) {
    if (topic == 'hermes/hotword/default/detected') {
        console.log("Hotword detected!");
    } else if (topic == 'hermes/intent/INTENT_NAME') {
        console.log("Intent detected!");
    }
});

Wildcard topics

We don’t have to manually subscribe to the topic for each intent. If we want to trigger an action when receiving any intent, we can simply subscribe to the wildcard topic hermes/intent/#:

client.on('connect', function () {
    // Subscribe to any topic starting with 'hermes/intent/'
    client.subscribe('hermes/intent/#');
});

Accessing intent parameters

In the following example, when receiving an intent, we parse the payload to extract the slots associated with our intent:

client.on('message', function (topic, message) {
    if (topic.startsWith('hermes/intent/INTENT_NAME')) {
        var payload = JSON.parse(message);
        var name = payload["intent"]["intentName"];
        var slots = payload["slots"];
        console.log(`Intent ${name} detected with slots ` +
            `{JSON.stringify(slots)}`);
    }
});

This completes the basic tutorial on how to write an MQTT client in Javascript that triggers actions upon receiving wakewords and intents from the Snips platform.