Latest news

How I built a fully functional Deep Learning Neural Network chatbot platform (NLU Engine) in under…

No comments

How I built a fully functional Deep Learning Neural Network chatbot platform (NLU Engine) in under a week….

Image Source: TechBubble Technologies EU

For the last year or so I have been working on a number of Artificial Intelligence programs, covering computer vision and natural linguistics amongst other projects. In the field of natural linguistics I have built two main projects, a chatbot engine similar to Chatfuel, and an Artificial Intelligence E-Commerce store which is an E-commerce application for businesses that is artificially intelligent and provides a speech based, and textual based interface that allows visitors to communicate with the store, find product information, navigate the store, and manage their cart and account. Recently we demonstrated the Artificial Intelligence E-Commerce Store as an Alpha Startup at Collison Conference in New Orleans which was a great experience and we received some great feedback and interest.

Up until recently both of the NLP projects had been built by creating a backend and frontend system that communicates with API.ai, recently acquired by Google. Through building these systems I began to learn more about how they work internally, understanding how intents, entities, synonyms and context works together to create a powerful NLU engine that can serve multiple chatbots. Anyone that knows me as a programmer, knows that I am never happy with using services, and always thrive to learn enough about services to build my own version, and that is exactly what I have done over the last week.

Through working with the API.ai platform, I had identified how these types of systems work, but also where I feel there could be improvements that would better the system. I had looked into RasaNLU as a possible solution to creating my own platform, but I felt it was very bulky and it also lacked features such as context management, intent actions and a few other features I believe are critical to an NLU engine. Having identified the basics of what I wanted to accomplish, I set about building my first attempt at an NLU engine.

The features I wanted to include were as follows:

  • Intent classification
  • Entity training / classification
  • Synonym training / classification
  • Context management
  • Intent actions
  • Trained / interfaced via API
  • Multiple models / users
  • Sentiment analysis
  • Webhooks
  • Management via TechBubble ARC

My first task was to create the training program that would train the model, for this I decided to use Tensorflow and TFLearn. This part was fairly straight forward and involved processing a json file containing intents, intent actions, and intent responses, splitting the processed data into training and testing data, passing the data through tflearn, defining the Deep Learning Neural Network, training with gradient descent, and finally saving the model for use in the next part of the program. I completed this part of the program in round about 130 lines of code and was ready to move on to the next part of the system.

Next up was to create the classifier that would provide the response back to the end user, again this part of the program took around 130 lines of codes, and I split the programming into 4 parts, intent classification first, then I worked on contexts, then entities and finally the server that would serve the API. At first I created this a simple command line program that would take inputs for model ID and user ID, load the relevant model and classify a hard coded response. Once that was working, I then sorted out the context management which was fairly straight forward and basically involved adding context in/out and clear fields to the training data, then keeping a persistant file with the current context states for each user. Once that was in place it was just a case of referencing that information against the data to respond accordingly.

For entity management I tried a couple of solutions including SpaCy and Mitie. Mitie seemed the best to suit my needs but I did come up against some issues with this. I built the entity training program which, like the intent training program at this point, was a command line application that took in a model ID and generated the relevant model. After this working on testing I began to notice that the entity returned in the examples was not actually returned from Mitie, but based on the token range of the entity value in the input. This threw me a bit, as entity management in API.ai includes an entity IE Product, a reference IE Computer and then synonyms IE PC, Computer, pc, computer etc. When an intent was classified the returned entity value would be the reference regardless of whether the entity was matched via a synonym. After a night of banging my head against the desk, I worked out a way of mapping the synonyms to the matched entity value and replacing this with the entity name, or reference.

With this in place, the next stage was to modify my trainer to replace annotations in the training data with a reference to the entity name, then retrain the model, once I added the NER (Named Entity Recognition) extractor to the classification pipeline, I simple replaced found entity values in the user input with the entity name, and bingo, the classifier would correctly identify intents that used entity references or synonyms.

The final stage was to add a simple Flask server that would serve up the core of the program and trigger either one of the two training applications or the intent classifier. Including training data the total amount of lines of code is less than 600 lines, which in comparison to other open source examples I have found that have half the features, is an extremely large improvement. It is not ready to be put into production yet as I have more features to include, a lot more training and testing to do etc, but in the space of less than a week I was able to build my own, fully functional NLU engine which is considerably smaller than any other solutions I have seen up until now.

Updated features so far include:

  • Trained / interfaced via API
  • Still trained from local Json files
  • Intent classification
  • Entity training / classification
  • Synonym training / classification
  • Context management
  • Intent Actions
  • Multiple models / users

Future features will include:

  • Sentiment analysis
  • Webhooks
  • Managed via TechBubble ARC

You can check out the photos on this page to see some screen shots of the progress, and I have also include a couple of videos of the original A.I. E-commerce Store for you to check out, you can also check out the videos gallery on the Artificial Intelligence E-Commerce demo site. Any comments or feedback, feel free to leave a message.

https://medium.com/media/8af32a6d8d97ed002a536bc743156c9f/hrefhttps://medium.com/media/765d4319430c69bbd8d67798176c5bed/href

Originally published at www.techbubble.info.

#ArtificialIntelligence #AI #DeepLearning #NeuralNetwork #NLU #Tesnorflow #TFLearn #TechBubble #TOA


How I built a fully functional Deep Learning Neural Network chatbot platform (NLU Engine) in under… was originally published in Chatbots Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.

Source: Chatbots Magazine

magnoliaHow I built a fully functional Deep Learning Neural Network chatbot platform (NLU Engine) in under…

Related Posts

Leave a Reply