Building Conversational Systems with NLP

• Conversational systems involves building systems capable of engaging in conversations with humans.
• Natural Language Processing (NLP) is a field of AI that allows computers to understand and process human language.
• Key NLP features for building conversational systems include:
  • Sentiment analysis.
  • Topic modeling.
  • Text categorization.
  • Text clustering.
  • Information extraction.
  • Relationship extraction.
  • Named entity resolution.

Rule-Based vs. AI Chatbots:

• Rule-based chatbots follow pre-defined paths and rules.
• AI chatbots use machine learning to understand user intent and generate responses.

Must-Have Chatbot Features:

• Human-like conversation that goes beyond answering questions to holding an intelligent conversation.
• Personalization where the bot brings in customer details and treats them individually.
• Version control for the chatbot system with rollback capabilities.
• Hybrid Model: Chatbot platforms are based on linguistic rules or AI/machine learning
  • Linguistic rule-based chatbots require humans to create applicable rules and responses.
  • AI/Machine learning-based chatbots needs data for curated training.
  • Hybrid model allows for the best of both worlds.
• The data is a key consideration in choosing a platform.
  • People reveal personal preferences, views, opinions, and feelings in everyday conversation.
  • Feedback into the conversation can increase user engagement with the chatbot
• Vendor lock-in: Select chatbots that can run on a variety of platforms.
  • Examining how much the build can be reused may save resources in the long term.
• Data Security is a key consideration of the chatbot, particularly when dealing with regulations and personal information.
  • Flexibility is essential across multiple geographies and legal requirements.
• Shared development capabilities: The platform should allow for joint collaborative work.
• Track-record: Ensuring that the platform has a successful track record in your industry.

Selecting a Chatbot Platform involves:

• Choosing an end-to-end platform for design, building, testing, and hosting.
• Supporting different forms of integration.
• Providing NLU (Natural Language Understanding) features from the platform.
• Data Management & Analytics.
• Conversational Systems combined with Enterprise Systems, Integration, and Chatbots enable Omni-channel communication.

Onboarding Chatbot Use Case:

• A chatbot can facilitate corporate onboarding by allowing new hires to ask questions to the messenger interface.
• HR chatbot needs to be effective, reliable and enjoyable for the end user.
• The chatbot can be developed using platforms such as dialogFlow, Chatfuel, and Drift.
• Chatbot should be tested by new employees, while sourcing data from ERP and systems integration.
• Chatbots that offer a positive experience increase familiarity in the workplace.

Chatbot Metrics:

• Metrics to measure chatbot effectiveness include active users, issues resolved without human intervention, employee productivity, and customer satisfaction.
• Other metrics include cost reduction, error rate and response time.

Challenges in Implementing Chatbots:

• Employee resistance to change, with fears of job replacement.
• Management concerns about impact on employee productivity.
• Security of customer interactions in the chatbot database.
• Changes in customer responses when interacting with chatbots.
• Failure to address unique user needs.
• Lack of feedback integration for continuous improvement.
• Globalization requires multilingual support and cultural understanding.

Reasons for Chatbot Project Failures:

• Insufficient training data, therefore it is important to remember that more data produces better results.
• Poor conversations arising from single interface.
• The wrong chatbot platform.
• No/minimum integration.
• Inadequate data management.
• Security taken lightly.

IKEA's Anna Chatbot:

• The chatbot was retired after 10 years due to not delivering customer satisfaction it was created for.
• The issues arose from the chatbot not sounding not being able to answer direct questions properly.

Key Principles of Communication in Chatbots:

• Intent.
• Entity.
• Slot.
• Context.
• Response.
• Parameter.
• Utterance.
• Intent refers to what the users speak about or want to do and includes examples of each intent.
• Entities are when something is said.
  • Types: System-S, Custom-C, Session-N.
  • System Entities include date, time, number, unit currency, percentage, address, phone number, email, and color.
  • Examples: dates/times, price/location, previous order.
• Dialog can be linear or non-linear.
  • Linear dialogs deal with service, date, and time.
  • Non-linear dialogs deal with real talk
• Entities can be later added to intents and can sometimes be required or not required.
• Slots filling is when the user does not provide required entities.
• Contexts are used in non-linear dialog, normally expiring in 20 minutes.
• Follow-up intent: Follows a specific intent.
• Fallback intent: Returns if users are confused.
• Fulfillment: Executes in the background.
• Response: Message to the user.
• Webhook: A server that provides service to the chatbot.
• Console.firebase.google.com is used to create a firebase database and link it to the chatbot.
• Parameters: Could be required or not required
• An utterance is anything someone says to the chatbot and may refer simply to the user input.

Utterances to "I DO NOT UNDERSTAND":

• Use 8-10 different ways of saying "I do not understand" such as:
  • I did not catch that.
  • My understanding of that is low.

To Wrap Things Up:

• Example Utterances:
  • send something.
  • send Alan a gift.
  • send Alan a box of Chocolate.