Aisera Analytics PDF

Aisera Analytics This guide describes the metrics you can configure and leverage using Aisera’s Analytics capability. May 2024 Confidential and Proprietary. Copyright © 2024 AISERA Inc. Al...

Aisera Analytics This guide describes the metrics you can configure and leverage using Aisera’s Analytics capability. May 2024 Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 Table of Contents Table of Contents..................................................................................................3 Overview............................................................................................................... 7 Introducing Aisera Analytics................................................................................. 7 Types of Dashboards........................................................................................................... 8 Requests..........................................................................................................................8 Sessions...........................................................................................................................9 Users.................................................................................................................................9 Knowledge....................................................................................................................... 9 Tickets..............................................................................................................................9 App Performance.......................................................................................................... 10 Business Metrics.......................................................................................................... 10 Tenant Configuration.........................................................................................................10 Resolution Rate.............................................................................................................10 Pre-Built Analytics...............................................................................................13 Requests.............................................................................................................................. 13 Total Requests.............................................................................................................. 13 Resolution Rate.............................................................................................................13 Not Helpful Rate............................................................................................................ 14 Abandoned Rate........................................................................................................... 14 Escalation Rate.............................................................................................................14 Assisted Rate.................................................................................................................14 Resolution Rate Over Time..........................................................................................15 Escalation Rate over Time........................................................................................... 15 Requests over Time......................................................................................................16 Resolved Requests by Category................................................................................16 Unresolved Requests by Category............................................................................ 17 Top Resolved Intents.................................................................................................... 17 Top Unresolved Intents................................................................................................ 17 Sessions............................................................................................................................... 18 Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 2  Total Sessions............................................................................................................... 18 ESAT................................................................................................................................ 18 Average Daily Sessions................................................................................................19 Live Sessions................................................................................................................. 19 Average Session Duration........................................................................................... 19 Total Request................................................................................................................ 19 Average Daily Requests............................................................................................... 19 Average Conversation Duration................................................................................. 19 Average Session Request........................................................................................... 19 Average User Session.................................................................................................. 19 Sessions Over Time..................................................................................................... 20 ESAT Over Time.............................................................................................................20 Requests by ESAT........................................................................................................ 20 ESAT surveys over Time............................................................................................... 21 Top Intents..................................................................................................................... 21 Intents Usage and Engagement................................................................................ 22 Flows Completion.........................................................................................................22 User...................................................................................................................................... 23 Total Active Users........................................................................................................ 23 New Users..................................................................................................................... 23 Live Users.......................................................................................................................24 VIP Users........................................................................................................................ 24 Total Active Users over Time...................................................................................... 24 Number of Requests by Top Users............................................................................24 Device Type................................................................................................................... 25 Device Platforms.......................................................................................................... 26 Web Page Sources....................................................................................................... 27 Browsers....................................................................................................................... 28 Knowledge...........................................................................................................................29 Total Knowledge document....................................................................................... 29 Knowledge Article Utilization Rate............................................................................30 Average Click-Through Rate.......................................................................................30 Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 3  Top Knowledge Gaps................................................................................................... 30 Top Documents in Requests by Click-Through.......................................................30 Ticket.................................................................................................................................... 31 Total Tickets...................................................................................................................31 Active Tickets................................................................................................................ 31 MTTR (Mean Time To Resolution)................................................................................... 31 Tickets over Time......................................................................................................... 32 MTTR over Time............................................................................................................ 32 Tickets by Priority over Time...................................................................................... 32 Tickets by Type............................................................................................................. 33 Tickets by Priority......................................................................................................... 33 App Performance............................................................................................................... 34 Total Intents.................................................................................................................. 34 Total Entities................................................................................................................. 34 Total Fulfillments.......................................................................................................... 34 Intents Utilization.........................................................................................................35 Entities Utilization........................................................................................................35 Fulfillments Utilization.................................................................................................35 Top Intents.....................................................................................................................35 Top Entities................................................................................................................... 36 Total Knowledge Articles.............................................................................................37 Total Conversation Flows............................................................................................ 37 Total Action Flows.........................................................................................................37 Total Service Catalog...................................................................................................38 Total Custom Messages............................................................................................. 38 Knowledge Articles Utilization.................................................................................. 38 Conversation Flows Utilization.................................................................................. 38 Action Flows Utilization............................................................................................... 38 Service Catalog Utilization......................................................................................... 38 Custom Messages Utilization.................................................................................... 38 Fulfillments Breakdown............................................................................................... 38 Request Fulfillments by Source.................................................................................39 Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 4  Top Fulfillments............................................................................................................ 40 Business Metrics................................................................................................................40 Total Time Saved for Employees................................................................................40 Agent Impact................................................................................................................. 41 Avoided Cost..................................................................................................................41 Custom Analytics................................................................................................42 Conversations by Channel......................................................................................... 42 Conversations by Country..........................................................................................43 Top Entities in Conversations.................................................................................... 43 Top Intents in Conversations..................................................................................... 44 Unresolved Analytics.......................................................................................... 44 Top Flow Gaps............................................................................................................... 45 Top Intents with KB Gaps............................................................................................46 Top Recommended Actions (Actions Recommended the Most)........................ 47 Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 5 Overview Aisera AISM provides advanced Reporting, Dashboards and Analytics capabilities, with pre-built reports & dashboards, as well as the ability to create custom reports & dashboards based on any available data in the platform. Some of the key metrics include resolution rate, escalation rate, abandoned rate, customer & employee satisfaction (CSAT/ESAT), MTTR, top unresolved intents, Knowledge Gaps, top intents by sentiments, etc. Introducing Aisera Analytics Aisera provides two types of Analytics capabilities: 1. Pre-Built Analytics 2. Custom Analytics Pre-Built Analytics are out-of-the-box (OOTB)dashboards that are available immediately so customers can derive insights and value as they go live. Navigation: Amdin UI → Analytics → Pre-Built Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 6 Pre-Built Analytics dashboards provide metrics related to Requests, Sessions, Users, App Performance and many more. Users can select various filters and can even save custom filters as per individual needs. Types of Dashboards Each of the tabs on the Pre-Built Analytics page is a dashboard that is collected or calculated by the Aisera platform. Some of the key dashboards include: Requests Sessions Users Knowledge Tickets App Performance Business Metrics Dashboard Description Name The Requests (Conversation Requests) are defined as a series Requests of messages exchanged between the Users and AiseraGPT application (App). Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 7  The Request dashboard captures the insights into the conversation requests that users have with the app. It focuses on metrics such as requests that were successfully answered (and not answered) by the app, escalated to agents, user drop-offs, intents that helped resolve the request and that did not help, identify knowledge gaps, etc. A session is considered to have started as soon as a user sends Sessions a message and ends when the user indicates that the problem is solved, abandons the chat, or is escalated to an agent. A session can have multiple conversation requests. Hence, the number of total requests >= no of sessions. The Sessions dashboard captures the insights of sessions created by users on the virtual assistant i.e. it displays insights into the conversation initiations by users when interacting with the virtual assistant in a selected time period. The Users dashboard captures insights on the type of users Users interacting with the virtual assistance and focuses on metrics such as who are the Top users, New users, Live users, VIP users (only applicable for ServiceNow), their Locations, platform..etc. These insights help understand user trends and focus on improving user experience thereby improving user retention. The Knowledge dashboard captures the count of knowledge Knowledge documents that were used in all conversations with corresponding Click-Through-Rate (CTR). Note: CTR is taken into consideration for events when/if the users have clicked on the document title or clicked on the “read more” option. The Tickets dashboard captures the insights of all the tickets Tickets that have been ingested from various endpoints into Aisera. It focuses on the metrics such as Total tickets created, MTTR, priorities, types..etc Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 8  The App Performance dashboard captures the overall insights App of the VA and is designed from a VA admin’s perspective. Performance It focuses on helping admins improve overall VA performance and captures matrices such as how many intents are active and helpful, total fulfilments and how many are active & helpful, conversation requests that were understood by the bot vs not understood..etc. The Business Metrics dashboard provides business users or Business Executives a comprehensive view of key indicators pertaining Metrics to User Productivity, Agent Impact and Avoided cost. These metrics are essential for enterprises to assess the value and determine ROIs. Tenant Configuration Resolution Rate One of the primary KPIs providing value to our customers is Resolution Rate. You are required to set up the Tenant Level Configuration, to capture data that the Aisera platform uses to calculate the Resolution Rate. To set up this configuration, navigate to Admin UI → Settings → Configuration. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 9 Within Configuration, select Resolution Rate as shown in the image below. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 10 Non-Flow Resolution considered as: Use non-flow resolution to use a default fulfillment status of “Resolved” or “Abandoned”. ○ If set to “Resolved”, the conversation will be considered as “Resolved” if the bot found the relevant Knowledge Article and served as the response. If the user explicitly clicks on the thumbs-down icon after Non-Flow Resolution is set to Resolved, the conversation will be considered as unresolved. ○ If set to “Abandoned”, the conversation will be considered as “Abandoned” if there is no response from the user. If the user explicitly clicks on the thumbs-up icon, the conversation will be considered resolved. If the user clicks on the thumbs-down icon, the conversation will be unresolved. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 11 Pre-Built Analytics The Pre-Built Analytics page consists of multiple OOTB dashboards that represent analytics data categories. This segment will discuss key metrics related to Requests Dashboard. Requests At the top of the Pre-Built Analytics page, you will see a section of Key Performance Indicators or analytics that are used across various dashboards. This will give you a foundational understanding of the Aisera analytics capabilities. Total Requests This metric captures the total number of conversation requests that users had with the chatbot. Resolution Rate Percentage of conversation requests that were successfully answered by AiseraGPT. This metric is calculated as the number of resolved and assisted requests divided by the number of relevant requests (resolved, unresolved, assisted, abandoned, unhandled). Note: A conversation is resolved if the user has clicked yes or thumbs up in the feedback Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 12  Answers are displayed only if the confidence score is above the threshold set by the Conversational AI model. Not Helpful Rate Percentage of conversation requests that did not have appropriate fulfillment. These are the conversations where the user has explicitly clicked no or thumbs down in the feedback, irrespective of the fulfillment provided by the chatbot. Note: Not helpful requests - Indicates requests where the bot found an intent, and responds back with the solution but the user clicks thumbs down. Abandoned Rate This metric captures the percentage of requests where users did not respond or dropped off within a conversation. Escalation Rate The percentage of total conversation requests that were not answered by the Aisera application or where the user specifically clicked “No” on the feedback option. This indicates the requests where users have given negative feedback to the bot response and then reached out to Support to create a ticket or give a live hand-off to the agent. Note: Escalated requests - Indicates requests where users have given negative feedback to the bot's response and then reached out to support or created a ticket. Assisted Rate This metric captures the requests where users have directly reached out to Support or created a ticket. This is calculated as the number of assisted requests divided by the number of relevant requests (resolved, unresolved, assisted, abandoned, unhandled). Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 13 Resolution Rate Over Time This metric shows the percentage of conversations between users and the chatbot and that the queries were answered and resolved by the chatbot without being escalated/transferred to Support Agents. Note: Answers are displayed only if the confidence score is above the threshold set by the Conversational AI model. It is possible to filter conversations by user feedback Escalation Rate over Time Percentage trend of conversation requests that were not answered by Aisera or where the user specifically clicked “No” on the feedback option for the selected time period. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 14 Requests over Time Breakdown of Resolved vs Unresolved conversation count over time. Resolved Requests by Category Breakdown of Resolved Conversations by Categories with the ability to drill down to sub-categories and intents. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 15 Unresolved Requests by Category Breakdown of Unresolved Conversations by Categories with the ability to drill down to sub-categories and intents. Top Resolved Intents Breakdown of top intents from resolved conversations. Top Unresolved Intents Breakdown of top intents from unresolved conversations. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 16 Sessions Total Sessions This metric captures the total number of user sessions that took place in a given time period. The session is considered to have started as soon as a user sends a message and ends when the user indicates that the problem is solved, abandons the chat, escalates to an agent or the session times-off due to user inactivity. ESAT This metric captures the average End User Satisfaction survey score for a given time period. This survey is triggered at the end of the session and can be configured as per customer requirement. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 17 Average Daily Sessions This metric captures the average number of daily sessions that took place. Live Sessions This metric captures the number of live sessions where the chatbot is being engaged. Average Session Duration This metric captures the time from the minute the user first engages with the chatbot to the minute there was no activity or the session times out. Total Request Captures the total number of conversations that took place. Average Daily Requests Captures the average number of daily conversations that took place. Average Conversation Duration Calculates and stores the average time from the beginning of all conversations to the end of all conversations. Average Session Request Captures the average number of conversations per session. Average User Session The number of conversation sessions that have occurred on average per user. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 18 Sessions Over Time Captures total count of user sessions over the selected time period. ESAT Over Time Captures the trend of the ESAT for the selected time period. Requests by ESAT Captures the ESAT score based on the number of conversations/requests. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 19 ESAT surveys over Time Captures the ESAT score trend for user sessions over time. Top Intents This metric shows the trend of Top Intents that have been discovered by the bot in the conversation between all the users and their interaction with the chatbot. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 20 Upon hovering the mouse on any of the intents, the graph shows the length of time that this intent was used, as a percentage of a given time period. When you click on a specific intent, the diagram displays additional details for that intent, such as the associated Fulfillment, Resolution, Fulfillment Source etc. Intents Usage and Engagement Captures and stores the number of intents per user session or request within a selected time period. Flows Completion Captures the details of the workflow/fulfillment and shows the number of Requests where the flow was completed or not completed. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 21 User This dashboard allows you to see how many users are engaging with the chatbot in real time or over a given time period. This can include new users, VIP users, and most active users (top users). Total Active Users Captures the count of total users that have engaged with the virtual assistant for a given time period. “Total Users” has been renamed to “Total Active Users”. New Users Captures the count of new users that have engaged with the virtual assistant for a given time period. This metric will only be populated if a user’s details do not already exist within the Aisera application when the user logs in to the bot. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 22 Live Users Captures the count of live users engaging with the virtual assistant. VIP Users Captures the total number of VIP users that have engaged with a virtual assistant for a given time period. This metric is specific to user data ingested from ServiceNow. This metric gets populated when the VIP users from ServiceNow initiate a conversation with the virtual assistant. Total Active Users over Time Captures the trend of users (Total and New users) interacting with the virtual assistant for a selected time period. Number of Requests by Top Users Captures breakdown of top users (users who have engaged the most) with the chatbot. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 23 User Locations Captures the number of users from various locations interacting with the virtual assistant during the selected time period. Device Type Captures the user device types that interact with the chatbot. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 24 Device Platforms Captures the user’s platform type that interacts with the virtual assistant. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 25 Web Page Sources Captures the web page data from users who are in conversation with the virtual assistant. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 26 Browsers Captures the user's browser type when they are interacting with the virtual assistant. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 27 Knowledge If you are using your chatbot to answer or deflect Support tickets, you can use the Knowledge Analytics dashboard to track document ingestion, document serving, click-through rates, knowledge gaps, and topics that are requested most often. Refer to the Ticket dashboards to see the metrics for ticket creation and closure. Total Knowledge document Captures the total number of documents that have been crawled/ingested from the Source system for the selected date range Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 28 Knowledge Article Utilization Rate Calculates and stores the percentage of the total number of Knowledge Articles that have been served by the bot at least once for the selected date range. Average Click-Through Rate Calculates and stores the average click-through rate based on the click-through rates of the articles that have been clicked by the users after they were served by the bot. Average Click Through Rate (CTR) = CTR for KB1 + CTR for KB2 +… CTR for KBN / Total KBs Top Knowledge Gaps Breakdown of top knowledge gaps from unresolved conversations. Top Documents in Requests by Click-Through Captures the number of Knowledge Documents that were used in all conversations with corresponding Click-Through-Rates (CTR). Note: CTR is assumed for events when the user has clicked on the document title or clicked on the “read more” option. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 29 Ticket If you are using your chatbot to answer or deflect Support tickets, you can use the Ticket Analytics dashboard to track total tickets and active tickets. Total Tickets Captures count of total tickets created for a given time period. Active Tickets Captures count of tickets created from an active system of record for a given time period. MTTR (Mean Time To Resolution) Captures mean time to resolution of tickets that includes the time from when the ticket was first submitted and recorded in the system of record to the time it was successfully resolved and marked in the system of record. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 30 Tickets over Time Captures ticket count trend over a period of time. MTTR over Time Captures the MTTR trend over a period of time. Tickets by Priority over Time Captures the number of tickets created in a selected time period, broken down by priorities. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 31 Tickets by Type Captures the number of tickets created, broken down by Ticket Type. Tickets by Priority Captures the number of tickets created, broken down by priorities, as defined in the system of record. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 32 App Performance Use the App Performance dashboard to track Intents, Entities, and Fulfillments, and Phrases. Total Intents This metric displays the total number of active intents in the application that correctly calculate (understand) the user's request. Upon clicking on the metric, you will see the list of all the Intents created in the system. Total Entities This metric shows the total number of Entities that have been created in the system that are active. Upon clicking the graphic for this field,, you will see the list of all the Entities created in your Aisera System. Total Fulfillments This metric displays the total number of Fulfillments created in the system that are active. Active fulfillments include action flows, knowledge flows, service catalog items, and custom messages in the application. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 33 Intents Utilization This metric presents the percentage of the total number of intents that have been utilized in all of the User Requests to date. Upon clicking on this metric, you will see the list of all the intents that have been utilized/invoked within the application. Entities Utilization This metric shows the percentage of the total number of Entities that have been utilized for all user requests to date. If you click on this metric, you will see the list of all the Entities that have been utilized/invoked within the application. Fulfillments Utilization This metric captures the percentage of the total action-flows/fulfilments that have been utilized in serving the requests. Top Intents These are the most used Intents in requests.This chart shows the top Intents that have been identified by the bot in all of the user conversations with the bot. If you hover the cursor over any of the intent bar graph, you will see the percentage of the total conversation time that the specific Intent was detected. If you click on a specific intent, you will see additional details for that Intent, such as related Fulfillment, Resolutions, Fulfillment Source etc. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 34 Top Entities These are the most used entities in requests.The chart displays the Top Entities that have been identified in all of the conversations between the users and the chatbot. Upon hovering the mouse on any of the Entities, you will see details like the name of the Entity and the number of requests it was identified in. Upon clicking on one of the Entities, you will see additional details like Fulfillment, Resolutions, Fulfillment Source etc. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 35 The following set of metrics are described below. Total Knowledge Articles This metric shows the total number of knowledge articles that were used in active fulfillments. Total Conversation Flows This metric shows the number of conversation flows used in active fulfillments. Total Action Flows This metric shows the number of action flows used in active fulfillments. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 36 Total Service Catalog This metric displays the number of service catalog items used in intents active fulfillments. Total Custom Messages This metric shows the number of custom messages used in intents active fulfillments. Knowledge Articles Utilization This metric gives you the percentage of the total number of Knowledge Articles utilized in the requests. Conversation Flows Utilization This metric shows the number of knowledge flows utilized in the requests. Action Flows Utilization This metric shows the number of action flows utilized in the requests. Service Catalog Utilization This metric shows the number of service catalog items utilized in the requests. Custom Messages Utilization This metric shows the number of custom messages utilized in the requests. Fulfillments Breakdown This pie chart displays the count as well as the percentage of the fulfillment breakdown that was either Supervised or Unsupervised. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 37 The total number of Fulfillments for each category is shown next to the name of the category. Request Fulfillments by Source This pie chart shows the percentage of total fulfillments per source. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 38 Top Fulfillments This pie chart displays the fulfillments that are used most often in conversation requests. Business Metrics The Business Metrics dashboard contains data that show you how the chatbot is creating efficiencies in your business practices and saving money for your company. Some of the key metrics are described below. Total Time Saved for Employees This metric shows you how the bot saves productive working time for employees who use the bot. Each issue resolved is assumed to save employees 60 minutes on average. Total Employee Productivity Gained = Number of issues resolved * 60 minutes/issue resolved Note: This metric is 60 minutes by default but can be re-configured in the Admin UI Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 39 Agent Impact Measures the amount of time saved or gained by Agents when their workload is decreased due to workload handled by the bot. Illustrates how the bot eliminates work for the service desk and does the equivalent work of full-time agents. Helps visualize how issues accelerated are trending over time (for the selected time period) Avoided Cost Also sometimes called, Opportunity Cost, the avoided cost metric captures avoided support costs for the organization. This value is calculated by multiplying Agent Cost by Agent Impact. The above business metrics have the following AiseraGPT Configuration: 1. Average amount of time to resolve an issue (minutes) 2. Agent Capacity = the number of issues resolved per day 3. Cost per issue Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 40 Custom Analytics The Custom Analytics page allows you to create your own reports and Key Performance Indicators (KPI)s. Some of the custom visualizations are described below. Conversations by Channel This metric gives you a graph of conversations broken down into the channels you have configured. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 41 Conversations by Country This metric is a visualization of the breakdown of conversations by the country where the conversation took place. Top Entities in Conversations This metric is presented as a graphic of top Entities discovered in conversations. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 42 Top Intents in Conversations Most-used Intents are presented in this metric as a breakdown of top intents discovered in conversations. Unresolved Analytics Unresolved analytics are critical for identifying the improvement areas within the conversations performed by the chatbot. These metrics are thus handled separately within the Aisera Admin UI, and can be accessed from the AI Workbench. Navigation: Admin UI → AI Workbench → Unresolved Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 43 Top Flow Gaps Visualize flows that were served as fulfillments in requests. These requests eventually get auto-categorized as Flow Gap Terminal Node/ Flow- Gap Non- Terminal Node. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 44 Top Intents with KB Gaps Visualize intents mapped to knowledge documents that were served as fulfillments in requests. These Requests eventually get auto-categorized as KB Gap. You can take 2 actions if your request is categorized as a KB Gap: 1. Create a new KB 2. Modify the existing KB Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 45 Top Recommended Actions (Actions Recommended the Most) Top recommended/suggested actions that you can take to address top gaps affecting the highest number of requests. Each of these recommended actions corresponds to a possible impact that you can make upon addressing the gaps- Flow Gaps & KB Gaps. Possible impact = Number of requests with a given gap / auto-tag / Total number of unresolved requests Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 46  4. Confidential and Proprietary. Copyright © 2024 AISERA Inc. All Rights Reserved. Email: [email protected] | (650) 667 4308 47