Getting started with conversational bots using Wit.ai

Bots. The word is everywhere and each week seems to bring a new project or piece of technology — from Facebook's new bot-building platform to Microsoft's Bot Framework to Taco Bell's bot for Slack. Journalism is not immune. Platforms like Quartz and Purple use bots to bring a conversational feel to news with a mobile apps and SMS interfaces while CNN’s bot will send you personalized news right through Facebook Messenger.

The potential appeal of bots lies with the ability of a news consumer to interact with them in a conversational way. But building a bot that can understand natural conversation is an intimidating challenge. Where to start?

One tool that might help is Wit.ai, which helps you quickly build apps people can talk to. It isolates all the complicated details, so that you can focuses on making something that functions, rather than writing new machine learning and natural language processing software.

So let’s talk about how to use Wit.ai to make your bots conversational.

Setting up your project


First, you’ll need to set up the basic structure for your bot. In my experience, Facebook’s Getting Started documentation is really helpful for setting up a simple functioning Messenger Bot using Node.js and Express. This Github tutorial provides an example of full working code, and if you want to build your bot with Python you can use this Flask example. There are also a lot of detailed Slack bot tutorials out there, like Scotch.io’s tutorial on how to build a Slack bot with Node.js.

Once you have the code for your bot setup, you’ll just need to deploy it so that Facebook or Slack can interact with it. One easy way to do this is to deploy the bot using Heroku.

Starting your app on Wit.ai


Now it’s time to dive into Wit.ai. Start by making an account and then create a new application.

Create a new application on Wit.ai for your project.

Once you’ve made your application, you will be directed to your app’s "Story" page. A Story is a way to help represent the types of conversations that people will be having with your bot. For example, let’s imagine we’re building a bot called Capital Cities Bot that will tell a user what the capital city of any country is. Our first story might look something like this:

Here, we lay out an example of what a user might say to this bot as a Story.

We then tell tell Wit.ai how the bot should respond. For our example, we'll have the Capital Cities Bot echo the location that is requested by the user. (This isn't a something you'd likely deploy, but it helps illustrate how Wit.ai works.)

Our first step is to capture the location entity. We can do this by highlighting the location in the sentence, and then clicking the “Add a new entity button." Notice that Wit already has a lot of entity presets listed for users out of the box, of which we can use the wit/location entity.

We can structure our response from the bot by clicking the “Bot says…” button on the right. Here, we can specify how we want the bot to respond. Since our bot is just echoing back the location for now, let’s have it say “You requested a capital for Nigeria." We can have Wit automatically include the captured location entity in our sentence by adding curly braces around the entity name.

By highlighting "Nigeria", we can mark it as a wit/location entity. Since our Wit.ai Story lays out how users will likely interact with our bot, this tells Wit.ai that it should look for location entities like "Nigeria" in the messages that are similar to this Story. Afterwards, we can add some details to the Say function so that Wit.ai knows how the bot should respond to the message. As an example, we can tell Wit.ai to echo back the location entity it found in the user's message.

We can chat with our Wit.ai service directly from the Wit.ai website by clicking “~”. Notice that we only wrote a story for Nigeria, but the Wit.ai can generalize our example to other countries that the user might enter as well.

Though we only trained our bot with the Nigeria example, Wit.ai is able generalize the concepts that we laid out in our Story to a variety of country names and question structures from the user.

Creating a user-defined entity


In the Capital Cities Bot example, we were able to use one of Wit.ai’s pre-defined entities to extract the location from the user’s message. Wit.ai has a pretty extensive list of predefined entities, but we're also able to define entities on our own.

To help understand user-defined entities, let’s make a new example called Feelings Bot that will send you a message to complement how you’re feeling. For example, if you’re feeling sad, you can send the bot “I’m feeling sad” and it’ll send you a sad message in response. As before, let’s start by having the bot echo back the feeling that is sent to it by a user.

To do this, we are going to need a new entity. Let’s call it Emotion. To setup a new entity, switch to the Understanding tab and type a training example into the “Try out an expression” field. For Feelings Bot, let’s type “I’m feeling sad.” We can tag “sad” and add a custom entity like so:

Using the Understanding tab on Wit.ai, we can start to create custom entities and train Wit.ai to discover them in a user's message.

We can continue to do this for different feelings – happy, excited, depressed, ecstatic, and so on – clicking “Validate” after each addition. Eventually, you’ll notice that Wit.ai will begin to automatically tag the feelings in the example expressions you enter. This is how we train Wit.ai to recognize the custom entity we made for our application.

Notice that Wit.ai has some specific “Search Strategies” that it uses to help train the entity. For our purposes, we’ll want “free-text” and “keywords” set, but other more complex expressions may require “trait” to be selected. You can read more about Wit.ai Search Strategies in their documentation about entities.

Now we can add a Story, and use our emotion entity. Notice how Wit.ai can now echo the feelings from user input.

Here we see that with one Story and a custom emotions entity, Wit.ai is already able to generalize and pick out the emotions in the messages we send. For testing, these emotions are just echoed back to the user.

Keeping conversational context


You may have also noticed the “merge” function on the right side of each Story. This function serves to add data to the overall context of the conversation. This allows the bot to remember previous entities that have been captured in earlier messages from the users. Using our Feelings Bot as an example, let’s say that the user first says “I’m feeling sad” and the bot responds with a sad comment. If the user says “Tell me something else”, then the Feelings Bot will need to remember the feeling that the user had specified in their first message so that it can send another sad comment.

To add data to the context, simply enter the entity name into the merge function. Now, this data will be stored and persisted through the duration of the conversation so that the bot can reference the information as need.

Integrating Wit.ai into your bot’s server


So far we’ve learned how to get a basic bot running from the linked tutorials, and we also know how use Wit.ai to layout some of the basics response patterns for our bots. But how do we put it all together?

The next step is to integrate Wit.ai into the code for your bot’s server. Wit.ai has well-documented open source libraries and SDKs for iOS, Ruby, Node.js, and Python which you can access at the Wit.ai Github page. Using these libraries will make it easier for your bot’s server to interact with the Wit.ai platform, and each repository will have documentation explaining how add Wit.ai to your server. Once you have the Wit.ai client added to the server code, you can forward messages that are captured by the server (from the users) and then forward them to Wit.ai using something like the client.messages() function for Node.js. Wit.ai will respond with the messages you specified in your Stories, as well as the context and extracted entities.

And that’s how you make a bot that you can talk to. Please share your thoughts and advice in the comments! Now go build Iron Man’s Jarvis, or something crazy like that.

About the author

Bomani McClendon

Student Fellow

Latest Posts

  • Introducing StorylineJS

    Today we're excited to release a new tool for storytellers.

    StorylineJS makes it easy to tell the story behind a dataset, without the need for programming or data visualization expertise. Just upload your data to Google Sheets, add two columns, and fill in the story on the rows you want to highlight. Set a few configuration options and you have an annotated chart, ready to embed on your website. (And did we mention, it looks great on phones?) As with all of our tools, simplicity...

    Continue Reading

  • Join us in October: NU hosts the Computation + Journalism 2017 symposium

    An exciting lineup of researchers, technologists and journalists will convene in October for Computation + Journalism Symposium 2017 at Northwestern University. Register now and book your hotel rooms for the event, which will take place on Friday, Oct. 13, and Saturday, Oct. 14 in Evanston, IL. Hotel room blocks near campus are filling up fast! Speakers will include: Ashwin Ram, who heads research and development for Amazon’s Alexa artificial intelligence (AI) agent, which powers the...

    Continue Reading

  • Bringing Historical Data to Census Reporter

    A Visualization and Research Review

    An Introduction Since Census Reporter’s launch in 2014, one of our most requested features has been the option to see historic census data. Journalists of all backgrounds have asked for a simplified way to get the long-term values they need from Census Reporter, whether it’s through our data section or directly from individual profile pages. Over the past few months I’ve been working to make that a reality. With invaluable feedback from many of you,......

    Continue Reading

  • How We Brought A Chatbot To Life

    Best Practice Guide

    A chatbot creates a unique user experience with many benefits. It gives the audience an opportunity to ask questions and get to know more about your organization. It allows you to collect valuable information from the audience. It can increase interaction time on your site. Bot prototype In the spring of 2017, our Knight Lab team examined the conversational user interface of Public Good Software’s chatbot, which is a chat-widget embedded within media partner sites.......

    Continue Reading

  • Stitching 360° Video

    For the time-being, footage filmed on most 360° cameras cannot be directly edited and uploaded for viewing immediately after capture. Different cameras have different methods of outputting footage, but usually each camera lens corresponds to a separate video file. These video files must be combined using “video stitching” software on a computer or phone before the video becomes one connected, viewable video. Garmin and other companies have recently demonstrated interest in creating cameras that stitch......

    Continue Reading

  • Publishing your 360° content

    Publishing can be confusing for aspiring 360° video storytellers. The lack of public information on platform viewership makes it nearly impossible to know where you can best reach your intended viewers, or even how much time and effort to devote to the creation of VR content. Numbers are hard to come by, but were more available in the beginning of 2016. At the time, most viewers encountered 360° video on Facebook. In February 2016, Facebook......

    Continue Reading

Storytelling Tools

We build easy-to-use tools that can help you tell better stories.

View More