Lecture 2: AI-Driven Recommendations in Orthopedic Surgery -uOttawa

FIRST DEPLOYMENT OF AI-DRIVEN RECOMMENDATIONS IN ORTHOPEDIC SURGERY Prof. Pascal Fallavollita Medical Education, Training, and Computer Assisted Interventions (METRICS) Lab | www.uOttawa.com Université d’Ottawa | University of Ottawa Update on the Adaptive Syllabus | www.uOttawa.com Université d’Ott...

FIRST DEPLOYMENT OF AI-DRIVEN RECOMMENDATIONS IN ORTHOPEDIC SURGERY Prof. Pascal Fallavollita Medical Education, Training, and Computer Assisted Interventions (METRICS) Lab | www.uOttawa.com Université d’Ottawa | University of Ottawa Update on the Adaptive Syllabus | www.uOttawa.com Université d’Ottawa | University of Ottawa Update on the Adaptive Syllabus | www.uOttawa.com Université d’Ottawa | University of Ottawa Update on the Adaptive Syllabus Augmented/Virtual/ Mixed Realities Medical Education History of Surgery Pre-operative solutions Intra-operative solutions 3D Printing/nano AI /Medical Robotics AI/Aging AI and more AI | www.uOttawa.com Université d’Ottawa | University of Ottawa Group Projects (70%) 7 minute presentations PPT on April 3 & 10. 1-2 people max/group will present in front of class. 2 pages WORD doc. – a summary of your research article. The 1st page is your own summary. The 2nd page is a chatgpt4 summary. | www.uOttawa.com Université d’Ottawa | University of Ottawa Group Projects (70%) 1. Find a journal publication on any Medical Technology in any capacity (diagnosis, pre-op, intraop, post-op, aging, cancer, biology, hospital, Health Canada, WHO, etc.) 2. Not a systematic review, not a literature review, not a state of the art, a pure research article that has a clear methodology and experimentation protocol with results. 3. Published 2022-onward 4. PubMed, MedLine, Google Scholar | www.uOttawa.com Université d’Ottawa | University of Ottawa Student Groups & Classroom Deadline: We finalize groups on Wednesday January 25. At the beginning of class please see me for those students not in a group All groups, please sit together during class DO NOT LEAVE AN EMPTY SEAT BETWEEN YOURSELVES THE CLASS IS AT FULL CAPACITY ~70 students | www.uOttawa.com Université d’Ottawa | University of Ottawa Live in dynamic city in terms of stakeholders policy makers can change what goes on anywhere, politics, healthcare, etc. important to have them at arms length CANADA’s CAPITAL CITY LARGEST BILIGUAL UNIVERSITY WORLDWIDE Fosters diversity “Silicon Valley” equivalent in ottawa good for applying for jobs 500 companies 21,000+ skilled workers Most advanced hospital competing w/ st/micahel in Toronto for being best hospital in canada around 800 beds lots of AI being implemented to optimize efficiency 2028/2029 Faculty of Health Sciences, 2023 FIRST DEPLOYMENT OF AI-DRIVEN RECOMMENDATIONS IN ORTHOPEDIC SURGERY Partnership w/ Ottawa hospital | www.uOttawa.com Université d’Ottawa | University of Ottawa During pandemic, most surgeries “Went down drain”; couldn’t integrate hospitals during virus human joint surgery | www.uOttawa.com Université d’Ottawa | University of Ottawa Provide me reasons as to why they are struggling struggling w/ knee snd hip replacement surgeries coming out of pandemic Pre-pandemic waitlist: there was a backlog when they were at full capacity; during covid they stopped these surgeries; now getting back into it there is even more people who need surgery lack of personnel, anaesthelogist, and nurses that left hospital or re-routed career | www.uOttawa.com Université d’Ottawa | University of Ottawa ARTHROPLASTY @ THE OTTAWA HOSPITAL (TOH) Issue everywhere across canada Government funded quality-based procedures every province has own legislation Arthroplasty hip and knee surgery for ex. Funded by Ontario government high-efficiency 4-joint operating room (OR) days (i.e., 4 joints within 8 hours 7:30am-3:30pm) and swing rooms (overlapping). Sub-par performance (2012- until Covid) Only 39% of arthroplasty days finish on time The average day ends 36 minutes late (4:03pm) 60% of the time, there was overtime and delays overtime and delay means more spending of hospital resources procedure continues past 3:30 pm | www.uOttawa.com Université d’Ottawa | University of Ottawa RESEARCH OBJECTIVES Research question: We aim to ensure highest Surgical Success Rate (SSR) using data analytics/ machine learning to find relevant parameters influencing SSR and team performance on efficiency….. Quantified *SSR is a critical metric that reflects the successful completion of 4-surgeries during an 8-hour timeframe with no overtime. binary - yes or no if yes, SSR = 100% if no, SSR does not equal 100% (ex. On Monday SSR = 20%, means that 80% of Mondays, we went into overtime (not good)) | www.uOttawa.com Université d’Ottawa | University of Ottawa RETROSPECTIVE DATA (4796 SURGERIES) From 2012-2020 5000 row by 40 columns excel data sheet each row equivalent to surgery each column is data parameter recorded Data prospectively recorded at TOH’s Surgical Information Management System (SIMS) & EPIC system Total Hip Replacement, Total Knee Replacement, Hip resurfacing, Unilateral knee replacement General and Civic campus 6 surgeons 44 nurses 52 anesthesiologists 1199 4-joint days 4796 patient surgeries log info for each surgery - recorded in real time for all surgeries happening at TOH Medical records platform | www.uOttawa.com Université d’Ottawa | University of Ottawa RETROSPECTIVE DATA PARAMETERS All data parameters saved for each surgery (columns) each row has 40 columns determining if patient had complications after surgery Number of days patient stays after surgery at hospital; usually 1-2 days In green: time stamps; can put value on it in terms of minute In red: surgery complications In purple: HCP/ team members In Pink: patient characteristics When did surgery end? Time it takes to clean./sterilze operating room between patients General or civic? Hip? Knee? When patient is first wheeled inside operating room When did surgery begin? Tells surgery team if you are healthy patient or not out of 4 (1-2 means healthy) if u have above, need to be careful during surgery | www.uOttawa.com Everything in green, and surgeon (purple) and patient demographics was analyzed Université d’Ottawa | University of Ottawa SURGEON EXPERIENCE / SURGICAL SUCCESS RATE experience Goes into overtime 60% of the time Most experienced surgeouns gap in SSR Surgeon experience – no correlation here with SSR 20 | www.uOttawa.com Université d’Ottawa | University of Ottawa DAY / SURGICAL SUCCESS RATE Teaching day Provide me a quick solution for TOH – what can we do with low SSR days? 21 | www.uOttawa.com monday - everyone complains another week begins friday - everyone looking at clock to begin weekend solution: spread surgeries evenly throughout week; move teaching day to Monday or Friday Université d’Ottawa | University of Ottawa GENDER/ SURGICAL SUCCESS RATE Potential reason as to why female patients have quicker surgical days with less overtime/delay? 22 | www.uOttawa.com Hypothesis: more difficult to do incisions on male compared to female due to muscle mass muscle mass less for females - easier incisions Université d’Ottawa | University of Ottawa FRAMEWORK FOR AI-BASED SURGICAL TRANSFORMATION (FAST) | www.uOttawa.com Université d’Ottawa | University of Ottawa Basic definition of machine learning “Holy grail” 24 | www.uOttawa.com way of describing the data machine learning has identified potential solutions and we can implement recommendations from AI solutions to change policies Université d’Ottawa | University of Ottawa We propose 3 frameworks (patient scheduling, team scheduling, workflow/time scheduling) Apply the machine learning process for each framework we need data - comes from SIM Ex. ML algorithm #10, #7, etc. Most robust model make actions based on it Make sure it is entered correctly, every cell and parameter had number to it, etc. missing data is an issue — results of ML won’t be robust 25 | www.uOttawa.com Continiually being developed many exist and it is up to us to apply a cohort and determine which one is the most robust and reliable and transparent which meets needs to alleviate overtime and delays Université d’Ottawa | University of Ottawa 1. PATIENT SCHEDULING FRAMEWORK decsion support system three frameworks (decision support systems) Patient demographic and team members have data of all patients how do we know which to schedule on what day to avoid overtime and delay | www.uOttawa.com Université d’Ottawa | University of Ottawa 1. PATIENT SCHEDULING FRAMEWORK age doesn’t play much of a factor as long as they have gd ASA when you are not as health, you are more subject to complications during surgery SSR becomes more complicated when you “introduce” males Highest SSR = 4 Healthy women and who are young Maybe schedule 4 surgeries w/ females for Monday and friday to have more success Lowest SSR = do not schedule the day with more than half of the patients as males ML corroborating and supported initial findings using statistics | www.uOttawa.com Université d’Ottawa | University of Ottawa 1. PATIENT SCHEDULING FRAMEWORK Don’t worry about models for midterm decision tree the least accurate model the most flexible to new data as ML models trained in patient scheduling data are least likely to overfit. undesirable machine learning behavior that occurs when the machine learning model gives accurate predictions for training data but not for new data. | www.uOttawa.com Université d’Ottawa | University of Ottawa 2. TEAM SCHEDULING FRAMEWORK Need to find combo of team members that produces at highest success rate comes down to respect, compromise, etc. | www.uOttawa.com Université d’Ottawa | University of Ottawa 2. TEAM SCHEDULING FRAMEWORK Nurse changed always gomna go to overtime if these ppl work tgt The significant impact that a single team member can have on the overall surgical success rate. Why is the team scheduling framework controversial? $$$$ talks for the staff statistics don’t always consider context - there could be external influences never gonna give “average” surgeons, nurses, etc. opportunity to improve / learn from those that are better — HR issue | www.uOttawa.com Cannot force/impose ppl to work w/ specific ppl — human resource issue Université d’Ottawa | University of Ottawa 2. TEAM SCHEDULING FRAMEWORK offers the most accurate results most prone to overfitting, especially if you choose the wrong ML model issue in algorithm development phase | www.uOttawa.com Université d’Ottawa | University of Ottawa 3. TIME MONITORING FRAMEWORK Can we find timestamp and can optimize/create criteria using ML (ex. Surgery should not be longer than 60 min) to improve SSR | www.uOttawa.com Université d’Ottawa | University of Ottawa 3. TIME MONITORING FRAMEWORK All numbers in minutes anaesthelogy preparation time prep time for med to be admitted to patient Anasethia finish time Only 5 parameters being optimized in this output Key recommendation is to lessen turnover time Turnover parameter is the time to clean/sterilize Ors between patient surgeries = can influence SSR Case parameter = we will not sacrifice patient safety by performing quicker procedures – even though SSR are the highest at 89% and 93% APT, Case (can’t rush surgery), and AFT; can change turnover time | www.uOttawa.com Université d’Ottawa | University of Ottawa 3. TIME MONITORING FRAMEWORK The most robust Medium accuracy | www.uOttawa.com Université d’Ottawa | University of Ottawa RETROSPECTIVE DATA OF 4796 SURGERIES - TIME MONITORING FRAMEWORK | www.uOttawa.com Université d’Ottawa | University of Ottawa Had highest impact on SSR can’t change policy and organizational since it is controversial least important Focus on this instead 36 | www.uOttawa.com Université d’Ottawa | University of Ottawa RETROSPECTIVE DATA PARAMETERS Recall that these are the time monitoring parameters (in mins) | www.uOttawa.com Université d’Ottawa | University of Ottawa DEFINITION OF TIME INTERVALS patient falls asleep Case start: first incision of the surgery Case finish: surgery is complete AFT: time anesthesia is done monitoring patient, usually done in the post-operative care unit APT: time from anesthesia start to anesthesia ready SPT: time between anesthesia ready and case start ↳ surgical preperation time Turnover: cleaning/sterilization time between surgeries | www.uOttawa.com Université d’Ottawa | University of Ottawa AI-DRIVEN TIME MONITORING BENCHMARKS All numbers in minutes The baseline was defined together with orthopedic surgeons as the most likely attainable in everyday clinical practice. AI recommendations which won’t affect patient safety during surgery Being late to these procedures is not helpful 39 | www.uOttawa.com Show up on time & Keep turnover time low Université d’Ottawa | University of Ottawa PROJECTED OVERTIME HOURS SAVED There exist solutions that offer 100% SSR combination of green timestamps being completed in time slots ML tells us irrespective of patient safety, etc. = 465 overtime hours saved = 9 minutes / surgery saved Saving 9 minutes per surgery during a 4 joint surgery day AI recommendations which won’t affect patient safety during surgery 40 | www.uOttawa.com Show up on time & Keep turnover time low Université d’Ottawa | University of Ottawa PROJECTED COST REDUCTION Average 60$ for each minute lost in surgery 4 joint surgeries in 1 OR 300 days per year operating room tells us ML works but w/ a caveat won’t discriminate if it will affect patient safety - issue if we do things properly, we can save money at TOH in division of orthopaedics # ORs used 9mins/surgery Cost Savings 1 2 $ 648,000 $1,296,000 5 10 41 | www.uOttawa.com If we use one solely for arthroplasty $3,240,000 $6,480,000 Université d’Ottawa | University of Ottawa EVALUATION OF AI RECOMMENDATIONS - PROSPECTIVE ARTHROPLASTY SURGERIES | www.uOttawa.com Université d’Ottawa | University of Ottawa Ontario is investing $300 million in 2023 as part of the surgical recovery strategy to increase scheduled surgeries and procedures In 2023 2 ORs 228 hip/knee surgeries…. every Saturday …. for 23 weeks…. weekend option possible if governemnt finances it Almost 6 months 43 | www.uOttawa.com Université d’Ottawa | University of Ottawa POSITIVE DEVIANCE SEMINARS What is Positive deviance seminars? = CHANGE IN A GODO WAY = foster healthy habits by introducing strategies that would foster change in a good way One or two ppl that are doing things efficiently all the time - need to know how they are doing this so we can implement these starts for entire group so they can collectively perform at highest level - these ppl r called positive deviant Ensure that the AI recommendations are followed Maybe bc their parameters were not going to be affected 44 | www.uOttawa.com  Surgeons/nurses participated x Anesthesiologists did not participate Université d’Ottawa | University of Ottawa POSITIVE DEVIANCE SEMINARS these prepositions help save time Improves quickness quicken surgery tasks Make sure nobody is standing doing nothing and that they are busy 45 | www.uOttawa.com Université d’Ottawa | University of Ottawa POSITIVE DEVIANCE SEMINARS be there when they fall asleep —> show up on time ***More important one ex. of incentivization: see kids, don’t have to pay for childcare 46 | www.uOttawa.com Université d’Ottawa | University of Ottawa OUR AI SOLUTIONS WORK We have achieved 93% SSR for the Saturday surgeries after positive deviance seminars and turnover motivation AI recommendations and PD seminars work In many cases, surgeons added a 5th and 6th arthroplasty procedure during the day with no overtime! (recall rule of 4 surgeries in 8 hour timeframe with no overtime) 47 | www.uOttawa.com Université d’Ottawa | University of Ottawa OUR AI SOLUTIONS WORK WHY DO YOU THINK IT WORKED WELL ON SATURDAYS??? 48 Conditions are ideal at the hospital (no interruptions, phone calls, emergencies | www.uOttawa.com Université d’Ottawa | University of Ottawa CONCLUSIONS Nursing unions cannot enter operating room before 7:30am Monday to Friday…….however on Saturdays this does not apply. Saturday surgeries minimal interruptions from other hospital entities or phone calls Other surgeries (i.e. general surgery) Are there other surgeries like arthroplasty where you need to complete a specific # of procedures per day Blueprint for AI driven recommendations in orthopaedics around the world how do we bring this in the weekdays when we are doing unions - much work left to be done 49 | www.uOttawa.com Université d’Ottawa | University of Ottawa