Listening to intents over MQTT using Python

In this tutorial, you will learn how to create a standalone Python application that listens to various events coming from the Snips Platform, in the form of MQTT messages.

In this tutorial, you will learn how to create a standalone Python application that listens to various events coming from the Snips Platform, in the form of MQTT messages.

The Hermes protocol

The Snips Hermes Protocol is the set of messages and exchange of messages between the various components of the Snips Platform. The messages are MQTT messages. The figure below describes the flow of messages during a simple voice interaction session.

You will most likely be interested in the following messages:

• hermes/hotword/<hotword_id>detected: when a wakeword is detected, this message will be sent, and Snips will listening to your voice command. You can for instance use this message to play a little tone, or start a led animation, indicating the start of a listening session

• hermes/asr/textCaptured: when listening, Snips will transcribe your query into text, and after a small period of silence, it will stop listening, and send this message. You may use this to play another tone, or stop a led animation, indicating end of listening

• hermes/intent/<intent_name>: after text has been captured, the NLU module will process the text and transform it into an intent, which is the final representation of your query that you can use as an actionable item. This is where you want to put your intent handler code

Listening to MQTT events

When launched, Snips starts an MQTT broker that it uses to broadcast the above MQTT messages. We can connect to this broker using the paho-mqtt library:

$ pip install paho-mqtt

The connection code looks as follows, replacing MQTT_BROKER_HOSTNAME with your broker hostname or IP:

import paho.mqtt.client as mqtt

HOST = 'raspi3-mika2.local'
PORT = 1883

def on_connect(client, userdata, flags, rc):
    print("Connected to {0} with result code {1}".format(HOST, rc))

def on_message(client, userdata, msg):
    print("Message received on topic {0}: {1}"\
        .format(msg.topic, msg.payload))

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect(HOST, PORT, 60)
client.loop_forever()

Detecting a wakeword

When a wakeword is detected, a message is sent on the hermes/hotword/<hotword_id>/detected topic, where hotword_id is the ID of your wakeword. The default value is default, which we will use here. We extend the above example to detect a wakeword and trigger an action as follows:

def on_connect(client, userdata, flags, rc):
    print("Connected to {0} with result code {1}".format(HOST, rc))
    # Subscribe to the hotword detected topic
    client.subscribe("hermes/hotword/default/detected")

def on_message(client, userdata, msg):
    if msg.topic == 'hermes/hotword/default/detected':
        print("Wakeword detected!")

Handling intents

We now extend our example to also trigger and action when an intent is detected by the NLU component. In this case, a message is sent on the hermes/intent/INTENT_NAMEtopic, where INTENT_NAME is the name of our intent.

def on_connect(client, userdata, flags, rc):
    print("Connected to {0} with result code {1}".format(HOST, rc))
    # Subscribe to the hotword detected topic
    client.subscribe("hermes/hotword/default/detected")
    # Subscribe to intent topic
    client.subscribe('hermes/intent/INTENT_NAME')
    
def on_message(client, userdata, msg):
    if msg.topic == 'hermes/hotword/default/detected':
        print("Hotword detected!")
    elif msg.topic == 'hermes/intent/INTENT_NAME':
        print("Intent detected!")

Wildcard topics

We don’t have to manually subscribe to the topic for each intent. If we want to trigger an action when receiving any intent, we can simply subscribe to the wildcard topic hermes/intent/#:

def on_connect(client, userdata, flags, rc):
    # Subscribe to any topic starting with 'hermes/intent/'
    client.subscribe('hermes/intent/#')

Accessing intent parameters

In the following example, when receiving an intent, we parse the payload to extract the slots associated with our intent:

import json

def on_message(client, userdata, msg):
    if msg.topic == 'hermes/intent/lightsTurnOnSet':
        payload = json.loads(msg.payload)
        name = payload["intent"]["intentName"]
        slots = payload["slots"]
        print("Intent {0} detected with slots {1}"\
            .format(name, slots))

This completes the basic tutorial on how to write an MQTT client in Python that triggers actions upon receiving wakewords and intents from the Snips platform.