Recently, there’s been a lot of fuss about chatbots. They’re ‘the new thing’ and anyone cool does something with AI in a project. Platforms like API.ai (Google) and Wit.ai (Facebook) are helping the hype along quite a bit. Within a few clicks, anyone can have a chatbot at their disposal. But how well do these chatbots appeal to the user? Do people actually have a complete conversation with one? Does anyone know? We should measure this…

How to measure?

We’ve been measuring website usage for over a decade. Why not do the same with this new technology? The go-to-tool for most people, when talking about measuring digital usage (web or app), is Google Analytics. Fortunately, Google Analytics and many other web trackers can be hooked up to almost anything. A chatbot is no exception. This article will go in-depth on the technical part of setting up a webhook for tracking analytics.

Webhooks

Most of these chatbot platforms support the use of webhooks. In our case we are using a webhook with API.ai for steering the conversation of the bot. Since we used the bot in Dutch language and the support for machine learning is awful at best, we’ve had to come up with some business rules to keep the conversation sane. For instance, the bot was interpreting ‘een’, a Dutch article, as a number 1, because ‘een’ also means 1 written in full.

The easiest solution is to write a Node.js application to catch these errors and correct them, before proceeding with the conversation. Since the bot is talking to a webhook now anyway, we could have the webhook fire analytics events.

Writing a Node webhook

Configuring a bot on API.ai is pretty straightforward and there is already a nice how-to guide for this on their website.

We need a simple node application for the platform to talk to. The easiest way is with Express. We are going to need some npm modules (express and body-parser), so run npm i --save express body-parser. The basic setup of any Express app probably looks like this.

const express = require('express');
  const app = express();
  app.use(require('body-parser').json());

  app.listen(process.env.PORT || 8080);
  

Your app will run on whatever is configured in system environment or on http://localhost:8080.

Next, you need an endpoint for the bot to make POST-requests. If you are familiar with Express, this should not be too difficult.

const router = express.Router();

  router.post('/webhook/ai', (req, res, next) => {
    // Do something with POST information
  });
  

Handling webhook calls

The data that the bot will send to you will be available in req.body. According to the docs of API.ai the format of the call to our webhook will contain the following:

{
     ...
      "sessionId": "1486656220806",
      "result": {
          "parameters": {
              "city": "Rome",
              "name": "Ana"
          },
          "contexts": [],
          "resolvedQuery": "my name is Ana and I live in Rome",
          "source": "agent",
          "score": 1.0,
          "speech": "",
          "fulfillment": {
              "messages": [
                  {
                      "speech": "Hi Ana! Nice to meet you!",
                      "type": 0
                  }
              ],
              "speech": "Hi Ana! Nice to meet you!"
          },
          "actionIncomplete": false,
          "action": "greetings",
          "metadata": {
              "intentId": "9f41ef7c-82fa-42a7-9a30-49a93e2c14d0",
              "webhookForSlotFillingUsed": "false",
              "intentName": "greetings",
              "webhookUsed": "true"
          }
      },
      ...
  }
  

So the ‘action’ that the bot wants us to do in the webhook is in the field with the same name: action. We can use this in our POST handler. You can configure actions throughout our bot and you are free to fill them in any Intent in API.ai. In this example the action is called ‘greetings’. In my case, if I only wanted to track the event in analytics, I could tag every Intent with an action called ‘track_event’.

router.post('/webhook/ai', (req, res, next) => {
    const action = req.body.results.action; // 'track_event'

    switch(action) {
      case: 'track_event':
        break;
    }
  });
  

Making the call to Google

The most straightforward way to trigger a Google Analytics event is to do a GET request.

First, we need to decide what to send to GA. These parameters will be included in our call. A good measurement in the case of API.ai is the current ‘Intent’ of the user. This gives a nice indication how a user has proceeded through the chat session, so we’ll include that.

It would also be good to have the session id of the current chat, so we can distinguish the path of a single user in our reporting.

We can easily compose a GET url for that. Since ‘Intent’ is included in req.body.result.metadata.intentName and API.ai provides us with req.body.sessionId.

const url = `https://www.google-analytics.com/collect?v=1&t=event&tid<INSERT_ACCOUNT_ID>&cid=${req.body.sessionId}&dh<INSERT_DOMAIN_HOST>&ec=Intent&ea=${req.body.result.metadata.intentName}&ev=1&aip=1`;
  

Note that we use ‘Intent’ as our event category (‘ec=’).

Now we can fire the request with a simple GET. I prefer using the request npm package for this. Make sure to run npm i --save request to get the package. Now we can use request.get.

  switch(action) {
      case: 'track_event':
        const url = `https://www.google-analytics.com/collect?v=1&t=event&tid<INSERT_ACCOUNT_ID>&cid=${req.body.sessionId}&dh<INSERT_DOMAIN_HOST>&ec=Intent&ea=${req.body.result.metadata.intentName}&ev=1&aip=1`;
        require('request').get(encodeURI(url))
          .then(err => {
            if (err) throw err;
            console.log('Successfully logged to GA');
          });
        break;
    }
  

So now we’ve logged our event in Google Analytics successfully, but the chatbot still expects a response from our webhook. If we don’t provide this, the bot will freeze up and stop responding to our user.

Providing feedback for the chatbot

In this case we are not adding information to the conversation. You could also enrich/alter messages that you want to send to the user at this point. For now, we only tracked our intent and we want to tell the bot to proceed with the conversation. We need at least the following information:

{
    messages: [],
    speech: '',
    displayText: '',
    contextOut: '',
    source: '',
  }
  

In the incoming request from the chatbot we have the required information available. We can put this right back in the response.

router.post('/webhook/ai', (req, res, next) => {
    const action = req.body.results.action; // 'track_event'
    switch(action) {
      case 'track_event':
        ...
        require('request').get(url)
          .then(err => {
            // Do something with err
            res.json({
              messages: req.body.result.fulfillment.messages,
              speech: req.body.result.fulfillment.speech,
              displayText: req.body.result.fulfillment.speech,
              contextOut: req.body.result.contexts,
              source: req.body.result.source
            });
          });
        break;

   ....
  

The bot will now respond to the user with the original fulfillment and stick to the user flow as if nothing happened. Except, now we know…

So, what’s next

As you can see, setting up a webhook for API.ai (or any platform for that matter) is extremely easy and it takes very little coding skills. Of course, you can expand this webhook to do all kinds of things. I’ll leave that to your own imagination. In our case, tracking a chatbot is already extremely useful. Now we know how our new toys are actually being used. For more information on how to analyze the resulting data, check out this article by my colleague Erik: ‘How to analyze chatbot data‘.