Virtual Assistant Metrics Quiz

Aisera Analytics Overview

• Aisera Analytics is a tool for configuring and leveraging metrics using Aisera's Analytics capability.
• The document is dated May 2024.
• AISERA is a company providing generative AI for businesses.

Table of Contents

• The document contains a comprehensive table of contents listing various topics and their respective page numbers.
• Topics include Overview, Introducing Aisera Analytics, Types of Dashboards, Requests, Sessions and more.
• The table of contents highlights pre-built analytics, requests, sessions, users, knowledge, tickets, app performance and business metrics.

Pre-Built Analytics

• Pre-Built Analytics pages contain OOTB Dashboards.
• Key dashboards included: Requests, Sessions, Users, Knowledge, Tickets, Application Performance, and Business Metrics.

Requests

• The Requests dashboard provides metrics on user-app interactions.
• Some key metrics are Resolution Rate, Not Helpful Rate, Abandoned Rate, Escalation Rate, Assisted Rate, and total requests.
• Resolution rate over time and requests over time are also relevant metrics.

Sessions

• The Sessions dashboard provides insights from user-virtual assistant interactions.
• Key metrics include total sessions, average daily sessions, live sessions, and average session duration.

Users

• The Users dashboard tracks user engagement with the virtual assistant.
• Metrics are focused on total users, new users, live users, VIP users and user locations.

Knowledge

• The Knowledge dashboard tracks knowledge document usage and click-through rates.
• Metrics include total knowledge documents and knowledge article utilization rate .

Tickets

• The Tickets dashboard monitors ticket-related metrics like total tickets, active tickets, and MTTR.
• Other metrics shown are mean time to resolution, ticket priorities and types.

App Performance

• The App Performance dashboard shows overall virtual assistant performance.
• Key metrics include active intents, entities, and fulfillments.

Business Metrics

• The Business Metrics dashboard displays key indicators for user productivity, agent impact, and avoided costs.

Tenant Configuration

• Resolution Rate is a key KPI for customers calculated on tenant-level configuration.
• Configuration is done through the Admin UI → Settings → Configuration.

Total Sessions

• Total sessions metric measures the total number of user sessions within a given time.

ESAT

• ESAT captures the average End User Satisfaction score within a specific time period.

Average Session Duration

• This captures time duration from user interaction to session end.

Total Requests

• This shows the total number of conversation requests.

Average Daily Requests

• This captures average daily number of conversation requests.

Average Conversation Duration

• This measures average duration of all conversations.

Average Session Requests

• This identifies average number of conversations per session.

Average User Session

• Shows the average number of conversations sessions.

Top Intents

• A metric to show the most frequently used intents.

Intents Usage and Engagement

• Shows intents per user session.

Flows Completion

• Shows the workflow details.

Total Intents

• The metric represents total active intents in the application.

Total Entities

• The metric indicates total active entities in the application.

Total Fulfillments

• The metric identifies total active fulfillments in the application.

Top Knowledge Articles

• The metric shows the total knowledge articles used in active fulfillments.

Total Conversation Flows

• The metric represents the number of conversation flows used in active fulfillments.

Top Entities

• The metric displays top utilized entities in user conversations.

Total Service Catalog

• The metric shows the number of service catalog items used in fulfilling intents.

Total Custom Messages

• The metric represents total custom messages utilized in fulfilling intents.

Knowledge Articles Utilization

• The percentage of used knowledge articles is shown.

Conversation Flows Utilization

• The metric shows the number of knowledge flows used in the requests.

Action Flows Utilization

• The metric shows how many action flows have been used.

Service Catalog Utilization

• The number of service catalog items used is listed.

Custom Messages Utilization

• The number of custom messages used is shown.

Fulfillment Breakdown

• The pie chart displays fulfillment breakdown, either Supervised or Unsupervised.

Total Fulfillments by Source

• The percentage of total fulfillments from each source is displayed.

Top Fulfillments

• The pie chart displays the most commonly used fulfillment types in conversations.

Business Metrics

• The metrics within this section show business efficiency in the use of the chatbot.
• Metrics include the cost and time saved for employees.

Agent Impact

• Measures the time saved by agents from reduced workloads.
• Illustrates work elimination via the chatbot.

Avoided Cost

• Measures the amount of support cost that has been reduced by the chatbot.

Custom Analytics

• Custom Analytics allows users to create their own reports and KPIs.
• Metrics include conversations, by different categories, like country or channel.

###Top Flow Gaps

• Visualizes flows served as fulfillments and categorizes them as terminal or non-terminal.

Top Intents with KB Gaps

• Shows intents mapped to knowledge docs served as fulfillment in requests.

Top Recommended Actions

• Top recommended actions to address knowledge gaps with the chatbot.