Virtual Assistant Metrics Quiz

Choose a study mode

Play Quiz
Study Flashcards
Spaced Repetition
Chat to Lesson

Podcast

Play an AI-generated podcast conversation about this lesson
Download our mobile app to listen on the go
Get App

Questions and Answers

What metric displays the number of service catalog items used in intents active fulfillments?

  • Total Action Flows
  • Total Conversation Flows
  • Total Custom Messages
  • Total Service Catalog (correct)

The Total Action Flows metric shows the number of knowledge articles utilized.

False (B)

How many minutes are assumed to be saved for employees per issue resolved by the bot?

60 minutes

The metric that provides the percentage of utilized knowledge articles is called _____ Articles Utilization.

<p>Knowledge</p> Signup and view all the answers

Match the following metrics with their descriptions:

<p>Total Time Saved for Employees = Shows time savings for employees per resolved issue Fulfillments Breakdown = Displays count and percentage of supervised vs. unsupervised fulfillments Action Flows Utilization = Shows number of action flows utilized in requests Top Fulfillments = Displays the most frequently used fulfillments in requests</p> Signup and view all the answers

What does the term 'Total Active Users' refer to?

<p>Users who have engaged with the virtual assistant during a time period (A)</p> Signup and view all the answers

The metric for New Users is populated when a user's details do exist in the Aisera application.

<p>False (B)</p> Signup and view all the answers

What captures the details of the workflow and shows how many requests were completed?

<p>Flows Completion</p> Signup and view all the answers

The metric that captures the count of users from various locations interacting with the virtual assistant is called _____

<p>User Locations</p> Signup and view all the answers

Match the metrics with their definitions:

<p>VIP Users = Total number of special users engaging with the assistant Live Users = Count of users engaging with the virtual assistant in real time Device Types = User devices that interact with the chatbot Number of Requests by Top Users = Breakdown of users who have engaged the most</p> Signup and view all the answers

What does the 'Total Active Users over Time' metric capture?

<p>The trend of users interacting with the assistant during a selected period (B)</p> Signup and view all the answers

The Device Platforms metric captures the devices used by users to contact the virtual assistant.

<p>True (A)</p> Signup and view all the answers

What does the Knowledge Analytics dashboard help track?

<p>Document ingestion and click-through rates (C)</p> Signup and view all the answers

The Knowledge Article Utilization Rate measures the total number of articles served by the bot.

<p>False (B)</p> Signup and view all the answers

What does CTR stand for in the context of Knowledge Article metrics?

<p>Click-Through Rate</p> Signup and view all the answers

The ______ captures the user's browser type when interacting with the virtual assistant.

<p>Browsers</p> Signup and view all the answers

Which of the following is not a metric tracked by the Knowledge Analytics dashboard?

<p>User email interactions (C)</p> Signup and view all the answers

Average Click-Through Rate is calculated by dividing the total CTR of articles by the number of articles.

<p>True (A)</p> Signup and view all the answers

What is meant by Top Knowledge Gaps in the context of unresolved conversations?

<p>It refers to the breakdown of knowledge gaps identified from conversations that were not resolved.</p> Signup and view all the answers

To improve the virtual assistant's performance, tracking ______ can highlight areas for enhancement.

<p>knowledge gaps</p> Signup and view all the answers

What does MTTR stand for?

<p>Mean Time To Resolution (B)</p> Signup and view all the answers

Active Tickets count includes all tickets created regardless of their status.

<p>False (B)</p> Signup and view all the answers

What does the Tickets by Priority measure?

<p>It captures the number of tickets created, broken down by priorities as defined in the system of record.</p> Signup and view all the answers

The _______ dashboard is used to track Intents, Entities, and Fulfillments.

<p>App Performance</p> Signup and view all the answers

Match the following ticket metrics with their descriptions:

<p>Total Tickets = Captures the count of total tickets created for a given time period. Active Tickets = Captures the count of tickets from an active system of record. Tickets over Time = Captures ticket count trend over a period of time. MTTR over Time = Captures the MTTR trend over a period of time.</p> Signup and view all the answers

Which of the following describes Total Intents?

<p>Total number of active intents that understand the user's request. (C)</p> Signup and view all the answers

The App Performance dashboard does not track Fulfillments.

<p>False (B)</p> Signup and view all the answers

What is indicated by MTTR over Time?

<p>It captures the Mean Time To Resolution trend over a period of time.</p> Signup and view all the answers

The dashboard used to analyze ticket trends is called the _______.

<p>Tickets over Time</p> Signup and view all the answers

What is Total Entities in the context of the App Performance dashboard?

<p>The total number of active Entities that have been created in the system (D)</p> Signup and view all the answers

What does the Intents Utilization metric represent?

<p>The percentage of total intents used in user requests (C)</p> Signup and view all the answers

The Top Intents chart displays the least used intents in requests.

<p>False (B)</p> Signup and view all the answers

What does the Entities Utilization metric indicate?

<p>It indicates the percentage of total entities that have been utilized for all user requests.</p> Signup and view all the answers

The _____ Utilization metric captures the percentage of action-flows that have been utilized in serving requests.

<p>Fulfillments</p> Signup and view all the answers

Match the following terms with their corresponding explanations:

<p>Intents Utilization = Percentage of total intents used in user requests Entities Utilization = Percentage of total entities used in user requests Top Intents = Most frequently used intents in conversations Fulfillments Utilization = Percentage of action flows utilized in requests</p> Signup and view all the answers

Which of the following provides additional details about an identified intent?

<p>Hovering over the intent bar graph (A)</p> Signup and view all the answers

Knowledge articles are not part of the active fulfillments.

<p>False (B)</p> Signup and view all the answers

What do users see when they click on the Top Entities chart?

<p>Additional details like the name of the entity and the number of requests it was identified in.</p> Signup and view all the answers

The metric for _____ shows the total number of knowledge articles used in active fulfillments.

<p>Total Knowledge Articles</p> Signup and view all the answers

Which metric would you check to see how many action flows have been utilized?

<p>Fulfillments Utilization (D)</p> Signup and view all the answers

Flashcards

Total Active Users

The total number of users who have interacted with the virtual assistant during a specific time period.

New Users

The count of new users who have interacted with the virtual assistant during a specific time period. This metric is only populated if a user's details aren't already in the system when they use the chatbot.

Live Users

The number of users currently interacting with the virtual assistant.

VIP Users

The total number of VIP users who have interacted with the virtual assistant during a specific time period. This metric is specific to users ingested from ServiceNow.

Signup and view all the flashcards

Total Active Users over Time

Shows the trend of users (total and new) interacting with the virtual assistant over a selected time period.

Signup and view all the flashcards

Number of Requests by Top Users

A breakdown of the top users (those who have interacted with the chatbot the most).

Signup and view all the flashcards

User Locations

Captures the number of users from different locations interacting with the virtual assistant during the selected time period.

Signup and view all the flashcards

Knowledge Analytics Dashboard

A feature within the bot's analytics that helps monitor how users are using knowledge articles. It tracks document ingestion, serving, click-through rate, and knowledge gaps.

Signup and view all the flashcards

Knowledge Article Utilization Rate

The percentage of Knowledge Articles that have been served by the bot at least once during the selected date range.

Signup and view all the flashcards

Average Click-Through Rate

The average click-through rate for all knowledge articles served by the bot.

Signup and view all the flashcards

Average Click Through Rate (CTR) Calculation

The average click-through rate of each knowledge article, divided by the total number of available articles.

Signup and view all the flashcards

Top Knowledge Gaps

Conversations where the bot was unable to provide a suitable answer, indicating knowledge gaps.

Signup and view all the flashcards

Top Documents in Requests by Click-Through

A view of the most frequently used knowledge documents, along with their corresponding click-through rates.

Signup and view all the flashcards

Total Knowledge Documents

The total number of documents that have been ingested into the bot's knowledge base.

Signup and view all the flashcards

Browser Information

The user's browser type, such as Chrome, Safari, or Firefox, is captured.

Signup and view all the flashcards

Web Page Sources

The bot captures the specific web pages that users view in their interactions.

Signup and view all the flashcards

Tickets over Time

Shows the trend of the number of total tickets created over a period of time.

Signup and view all the flashcards

MTTR over Time

Shows the trend of the average time it takes to resolve a ticket over time.

Signup and view all the flashcards

Tickets by Priority over Time

Shows the number of tickets created in a selected time period, broken down by priority levels.

Signup and view all the flashcards

Tickets by Type

Captures the number of tickets created, categorized by their type (e.g. bug, feature request).

Signup and view all the flashcards

Tickets by Priority

Shows the number of tickets created categorized by priority levels, as defined in the system used for managing tickets.

Signup and view all the flashcards

Total Intents

Measures the number of active intents in a chatbot application that can correctly interpret the user's request.

Signup and view all the flashcards

Total Entities

Shows the total number of active entities, which are the key information elements used in the chatbot's knowledge.

Signup and view all the flashcards

Total Fulfillments

Shows the number of active 'fulfillments' in the chatbot. Fulfillments are the actions that happen after the chatbot understands the user's request, like providing information or triggering a process.

Signup and view all the flashcards

MTTR (Mean Time To Resolution)

Measures the average time it takes to resolve a ticket, from when it's created to when it's marked as resolved in the ticketing system.

Signup and view all the flashcards

Active Tickets

Measures the number of active tickets created for a specific time period.

Signup and view all the flashcards

Total Conversation Flows

This metric measures the efficiency of using chatbot conversation flows by tracking the number of flows actively used in fulfilling user requests.

Signup and view all the flashcards

Total Action Flows

This metric tracks the number of action flows (pre-defined steps) actively used to fulfill user request.

Signup and view all the flashcards

Total Time Saved for Employees

The "Total Time Saved for Employees" metric shows how much time the chatbot helps employees save by resolving issues automatically.

Signup and view all the flashcards

Knowledge Articles Utilization

This metric tracks the number of knowledge articles used to fulfill user requests.

Signup and view all the flashcards

Total Service Catalog

This metric tracks the number of different service catalog items (like products or services) used to fulfill requests.

Signup and view all the flashcards

Intents Utilization

A measure of how many different Intents have been used in all user interactions with the chatbot.

Signup and view all the flashcards

Entities Utilization

Shows the percentage of Entities that have been used in user requests. Entities are like specific pieces of information mentioned by users.

Signup and view all the flashcards

Fulfillments Utilization

Represents the percentage of action flows or fulfillments that have been used to serve user requests. Fulfillments are the actions the bot takes based on user intents.

Signup and view all the flashcards

Top Intents

The top Intents most frequently used by users in conversations with the chatbot. Hovering over an intent bar shows its percentage of conversation time.

Signup and view all the flashcards

Total Knowledge Articles

The total number of knowledge articles used in actively serving user requests using the chatbot.

Signup and view all the flashcards

Study Notes

Aisera Analytics Overview

  • Aisera Analytics is a tool for configuring and leveraging metrics using Aisera's Analytics capability.
  • The document is dated May 2024.
  • AISERA is a company providing generative AI for businesses.

Table of Contents

  • The document contains a comprehensive table of contents listing various topics and their respective page numbers.
  • Topics include Overview, Introducing Aisera Analytics, Types of Dashboards, Requests, Sessions and more.
  • The table of contents highlights pre-built analytics, requests, sessions, users, knowledge, tickets, app performance and business metrics.

Pre-Built Analytics

  • Pre-Built Analytics pages contain OOTB Dashboards.
  • Key dashboards included: Requests, Sessions, Users, Knowledge, Tickets, Application Performance, and Business Metrics.

Requests

  • The Requests dashboard provides metrics on user-app interactions.
  • Some key metrics are Resolution Rate, Not Helpful Rate, Abandoned Rate, Escalation Rate, Assisted Rate, and total requests.
  • Resolution rate over time and requests over time are also relevant metrics.

Sessions

  • The Sessions dashboard provides insights from user-virtual assistant interactions.
  • Key metrics include total sessions, average daily sessions, live sessions, and average session duration.

Users

  • The Users dashboard tracks user engagement with the virtual assistant.
  • Metrics are focused on total users, new users, live users, VIP users and user locations.

Knowledge

  • The Knowledge dashboard tracks knowledge document usage and click-through rates.
  • Metrics include total knowledge documents and knowledge article utilization rate .

Tickets

  • The Tickets dashboard monitors ticket-related metrics like total tickets, active tickets, and MTTR.
  • Other metrics shown are mean time to resolution, ticket priorities and types.

App Performance

  • The App Performance dashboard shows overall virtual assistant performance.
  • Key metrics include active intents, entities, and fulfillments.

Business Metrics

  • The Business Metrics dashboard displays key indicators for user productivity, agent impact, and avoided costs.

Tenant Configuration

  • Resolution Rate is a key KPI for customers calculated on tenant-level configuration.
  • Configuration is done through the Admin UI → Settings → Configuration.

Total Sessions

  • Total sessions metric measures the total number of user sessions within a given time.

ESAT

  • ESAT captures the average End User Satisfaction score within a specific time period.

Average Session Duration

  • This captures time duration from user interaction to session end.

Total Requests

  • This shows the total number of conversation requests.

Average Daily Requests

  • This captures average daily number of conversation requests.

Average Conversation Duration

  • This measures average duration of all conversations.

Average Session Requests

  • This identifies average number of conversations per session.

Average User Session

  • Shows the average number of conversations sessions.

Top Intents

  • A metric to show the most frequently used intents.

Intents Usage and Engagement

  • Shows intents per user session.

Flows Completion

  • Shows the workflow details.

Total Intents

  • The metric represents total active intents in the application.

Total Entities

  • The metric indicates total active entities in the application.

Total Fulfillments

  • The metric identifies total active fulfillments in the application.

Top Knowledge Articles

  • The metric shows the total knowledge articles used in active fulfillments.

Total Conversation Flows

  • The metric represents the number of conversation flows used in active fulfillments.

Top Entities

  • The metric displays top utilized entities in user conversations.

Total Service Catalog

  • The metric shows the number of service catalog items used in fulfilling intents.

Total Custom Messages

  • The metric represents total custom messages utilized in fulfilling intents.

Knowledge Articles Utilization

  • The percentage of used knowledge articles is shown.

Conversation Flows Utilization

  • The metric shows the number of knowledge flows used in the requests.

Action Flows Utilization

  • The metric shows how many action flows have been used.

Service Catalog Utilization

  • The number of service catalog items used is listed.

Custom Messages Utilization

  • The number of custom messages used is shown.

Fulfillment Breakdown

  • The pie chart displays fulfillment breakdown, either Supervised or Unsupervised.

Total Fulfillments by Source

  • The percentage of total fulfillments from each source is displayed.

Top Fulfillments

  • The pie chart displays the most commonly used fulfillment types in conversations.

Business Metrics

  • The metrics within this section show business efficiency in the use of the chatbot.
  • Metrics include the cost and time saved for employees.

Agent Impact

  • Measures the time saved by agents from reduced workloads.
  • Illustrates work elimination via the chatbot.

Avoided Cost

  • Measures the amount of support cost that has been reduced by the chatbot.

Custom Analytics

  • Custom Analytics allows users to create their own reports and KPIs.
  • Metrics include conversations, by different categories, like country or channel.

###Top Flow Gaps

  • Visualizes flows served as fulfillments and categorizes them as terminal or non-terminal.

Top Intents with KB Gaps

  • Shows intents mapped to knowledge docs served as fulfillment in requests.
  • Top recommended actions to address knowledge gaps with the chatbot.

Studying That Suits You

Use AI to generate personalized quizzes and flashcards to suit your learning preferences.

Quiz Team

Related Documents

Aisera Analytics PDF

More Like This

Use Quizgecko on...
Browser
Open
Browser
Browser