Airbnb's Party Pooper Reduced Parties by 55% in Two Years PDF

8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years LIVESTREAM ...

8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years LIVESTREAM  SIGN IN TECH Meet Airbnb’s official party pooper, who reduced partying by 55% in two years P U B L I S H E D T U E , S E P 1 9 2 0 2 3 1 : 0 8 P M E DT U P DAT E D T U E , S E P 1 9 2 0 2 3 4 : 2 3 P M E DT Hayden Field WATCH LIVE @ H AY D E N F I E L D KEY POINTS As the person in charge of Airbnb’s worldwide ban on parties, Naba Banerjee has spent more than three years figuring out how to battle party “collusion” by users, flag “repeat party houses” and, most of all, design an anti-party AI system. The AI models look at hundreds of factors, including the reservation’s closeness to the user’s birthday, the user’s age, length of stay, the listing’s proximity to where the user is based, weekend vs. weekday, and whether the listing is in a popular location. There was a global 55% drop in parties reported on Airbnb between August 2020 and August 2022, according to the company, and since the worldwide launch of Banerjee’s system in May, more than 320,000 guests have been blocked or redirected from booking attempts. Follow your favorite stocks CREATE FREE ACCOUNT In this article ABNB +0.15 (+0.13%)  MARKETS  VIDEO  WATC H L I S T  MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 1/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years LIVESTREAM SIGN IN Naba Banerjee, Airbnb Source: Prashant Joshi | Airbnb Naba Banerjee is a proud party pooper. As the person in charge of Airbnb’s worldwide ban on parties, she’s spent more than three years figuring out how to battle party “collusion” by users, flag “repeat party houses” and, most of all, design an anti-party AI system with enough training data to halt high-risk reservations before the offender even gets to the checkout page. It’s been a bit like a game of whack-a-mole: Whenever Banerjee’s algorithms flag some concerns, new ones pop up. Airbnb defines a party as a gathering that occurs at an Airbnb listing and “causes significant disruption to neighbors and the surrounding community,” according to a company rep. To determine violations, the company considers whether the gathering is an open-invite one, and whether it involves excessive noise, trash, visitors, parking issues for neighbors, and other factors. Banerjee joined the company’s trust and safety team in May 2020 and now runs that group. In her short time at the company, she’s overseen a ban on high-risk reservations by users under age 25, a pilot program for anti-party AI in Australia, heightened defenses on holiday weekends, a  MARKETS  VIDEO  WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 2/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years host insurance policy worth millions of dollars, and this summer, a global rollout of Airbnb’s LIVESTREAM SIGN IN reservation screening system. Some measures have worked better than others, but the company says party reports dropped 55% between August 2020 and August 2022 — and since the worldwide launch of Banerjee’s system in May, more than 320,000 guests have been blocked or redirected from booking attempts on Airbnb. Overall, the company’s business is getting stronger as the post-pandemic travel boom starts to fade. Last month, the company reported earnings that beat analysts’ expectations on earnings per share and revenue, with the latter growing 18% year over year, despite fewer-than-expected numbers of nights and experiences booked via the platform. Turning parental party radar into an algorithm Courtesy: Airbnb Airbnb says the pandemic and hosts’ fears of property damage are the main drivers behind its anti-party push, but there have been darker incidents as well. A Halloween party at an Airbnb in 2019 left five people dead. This year between Memorial Day and Labor Day weekends, at least five people were killed at parties hosted at Airbnbs. In June, the  MARKETS  VIDEO  WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 3/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years company was sued by a family who lost their 18-year-old son in a shooting at aLIVESTREAM 2021 Airbnb SIGN IN party. When Banerjee first joined Airbnb’s trust team in summer 2020, she recalls people around her asking, “How do you solve this problem?” The stream of questions, from people above and below her on the corporate ladder, contributed to her anxiety. Airbnb’s party problem was complex, and in some ways, she didn’t know where to start. As a mother of five, Banerjee knows how to sniff out a secretive shindig. Last summer, Banerjee’s 17-year-old daughter had a friend who wanted to throw an 18th birthday party — and she was thinking about booking an Airbnb to do it. Banerjee recalls her daughter telling her about the plan, asking her whether she should tell her friend not to book an Airbnb because of the AI safeguards. The friend ended up throwing the party at her own home. “Being a mother of teenagers and seeing teenage friends of my kids, your antenna is especially sharp and you have a radar for, ‘Oh my God, okay, this is a party about to happen,’” Banerjee said. “Between our data scientists and our machine learning engineers and us, we started looking at these signals.” For Banerjee, it was about translating that antenna into a usable algorithm. In an April 2020 meeting with Nate Blecharczyk, the company’s co-founder and chief strategy officer, Banerjee recalls strategizing about ways to fix Airbnb’s party problem on three different time scales: “right now,” within a year, and in the general future. For the “right now” scale, they talked about looking at platform data, studying the patterns and signals for current party reports, and seeing how those puzzle pieces align. The first step, in July 2020, was rolling out a ban on high-risk reservations by users under age 25, especially those who either didn’t have much history on the platform or who didn’t have good reviews from hosts. Although Airbnb says that move blocked or redirected “thousands” of guests globally, Banerjee still saw users trying to get around the ban by having an older friend or relative book the reservation for them. Two months later, Airbnb announced a “global party ban,” but that was mostly lip service — at least, until they had the technology to back it up. Around the same time, Banerjee sent out a series of invitations. Rather than to a party, they were invites to attend party risk reduction workshops, sent to Airbnb designers, data scientists,  machine MARKETS  learning engineers and members  of the operations and communications VIDEO teams. In WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 4/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years Zoom meetings, they looked at results from the booking ban for guests under 25 and started LIVESTREAM SIGN IN putting further plans in motion: Banerjee’s team created a 24/7 safety line for hosts, rolled out a neighborhood support line, and staffed up the customer support call center. One of the biggest changes, though, was to remove the option for hosts to list their home as available for gatherings of more than 16 people. Courtesy: Airbnb Now that they had a significant amount of data on how potential partiers might act, Banerjee’s team had a new goal: Build the AI equivalent of a neighbor checking on the house when the high- schooler’s parents leave them home alone for the weekend. Around January 2021, Banerjee recalled hearing from Airbnb’s Australia offices that disruptive parties at Airbnbs were up and coming, just like they were in North America, as travel had come to a relative standstill and Covid was in full swing. Banerjee considered rolling out the under-25 ban in Australia, but after chatting with Blecharczyk, she decided to experiment with a party- banning machine-learning model instead. But Banerjee was nervous. Soon after, she phoned her father in Kolkata, India — it was between 10 p.m. and 11 p.m. for her, which was mid-morning for him. She is the first female engineer in  MARKETS  VIDEO  WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 5/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years her family, she said, and her father is one of her biggest supporters; he is typically the person she LIVESTREAM SIGN IN calls during the most difficult moments of her life. “I remember talking to him, saying, ‘I’m just very scared — I feel like I’m on the verge of doing one of the most important things of my career, but I still don’t know if we are going to succeed,’” Banerjee said. ”‘We have the pandemic going on, the business is hurting... We have something that we think is going to be great, but we don’t know yet. I’m just on this verge of uncertainty, and it just makes me really nervous.’” Banerjee recalled her father telling her that this has happened to her before and that she’d succeed again. He’d be more worried, he told her, if she were overconfident. In October 2021, Banerjee’s team rolled out the pilot program for their reservation screening AI in Australia. The company saw a 35% drop in parties between regions of the country that had the program versus those that did not. The team spent months analyzing the results and upgraded the system with more data, as well as safety and property damage incidents and records of user collusion. How the AI system works to stop parties Listings on Airbnb Source: Airbnb Imagine you’re a 21-year-old planning a Halloween party in your hometown. Your plan: Book an  house for one night, send  Airbnb out the “BYOB” texts and try to  avoid posting cliched Instagram MARKETS VIDEO WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 6/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years captions. LIVESTREAM SIGN IN There’s just one problem: Airbnb’s AI system is working against you from the second you sign on. The party-banning algorithm looks at hundreds of factors, including the reservation’s closeness to the user’s birthday, the user’s age, length of stay, the listing’s proximity to where the user is based, how far in advance the reservation is being made, weekend vs. weekday, the type of listing and whether the listing is in a heavily crowded location rather than a rural one. Deep learning is a subset of machine learning that uses neural networks — that is, the systems process information in a way inspired by the human brain. The systems are certainly not functionally comparable to the human brain, but they do follow the pattern of learning by example. In the case of Airbnb, one model focuses specifically on the risk of parties, while another focuses on property damage, for instance. “When we started looking at the data, we found that in most cases, we were noticing that these were bookings that were made extremely last-minute, potentially by a guest account that was created at the last minute, and then a booking was made for a potential party weekend such as New Year’s Eve or Halloween, and they would book an entire home for maybe one night,” Banerjee told CNBC. “And if you looked at where the guest actually lived, that was really in close proximity to where the listing was getting booked.” After the models do their analysis, the system assigns every reservation a party risk. Depending on the risk tolerance that Airbnb has assigned for that country or area, the reservation will either be banned or greenlit. The team also introduced “heightened party defenses” for holiday weekends such as the Fourth of July, Halloween and New Year’s Eve.  MARKETS  VIDEO  WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 7/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years LIVESTREAM SIGN IN Airbnb’s reservation screening system in action. Source: Airbnb In some cases, like when the right decision isn’t quite clear, reservation requests are flagged for human review, and those human agents can look at the message thread to gauge party risk. But the company is also “starting to invest in a huge way” in large language models, or LLMs, for content understanding, to help understand party incidents and fraud, Banerjee said. “The LLM trend is something that if you are not on that train, it’s like missing out on the internet,” Banerjee told CNBC.  MARKETS  VIDEO  WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 8/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years Banerjee said her team has seen a higher risk of parties in the U.S. and Canada,LIVESTREAM and the next- SIGN IN riskiest would probably be Australia and certain European countries. In Asia, reservations seem to be considerably less risky. The algorithms are trained partly on tickets labeled as parties or property damage, as well as hypothetical incidents and past ones that occurred before the system went live to see if it would have flagged them. They’re also trained on what “good” guest behavior looks like, such as someone who checks in and out on time, leaves a review on time, and has no incidents on the platform. But like many forms of AI training data, the idea of “good” guests is ripe for bias. Airbnb has introduced anti-discrimination experiments in the past, such as hiding guests’ photos, preventing hosts from viewing a guest’s full name before the booking is confirmed, and introducing a Smart Pricing tool to help address earnings disparities, although the latter unwittingly ended up widening the gap. Airbnb said its reservation-screening AI has been evaluated by the company’s anti- discrimination team and that the company often tests the system in areas such as precision and recall. Going global  MARKETS  VIDEO  WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 9/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years Courtesy: Airbnb LIVESTREAM SIGN IN Almost exactly one year ago, Banerjee was at a plant nursery with her husband and mother-in- law when she received a call from Airbnb CEO Brian Chesky. She thought he’d be calling about the results of the Australia pilot program, but instead he asked her about trust in the platform. Given all the talk she did about machine-learning models and features, she recalled him asking her, would she feel safe sending one of her college-bound kids to stay at an Airbnb — and if not, what would make her feel safe? That phone call ultimately resulted in the decision to expand Banerjee’s team’s reservation screening AI worldwide the following spring. Things kicked into high gear with TV spots for Banerjee, some of which she saw on the gym television between pull-ups. She asked her daughter for advice on what to wear. The next thing she knew, the team was getting ready for a live demo of the reservation screening AI with Chesky. Banerjee was nervous. The team sat down with Chesky after working with front-end engineers to create a fake party risk, showing someone booking an entire mansion during a holiday weekend at the last minute and seeing if the model would flag it in real time. It worked. Chesky’s only feedback, Banerjee recalled, was to change the existing message — “Your reservation cannot be completed at this point in time because we detect a party risk” — to be more customer-friendly, potentially offering an option to appeal or book a different weekend. They followed his advice. Now, the message reads, “The details of this reservation indicate it could lead to an unauthorized party in the home. You still have the option to book a hotel or a private room, or you can contact us with any questions.” Banerjee remembers a frenzy of activity over the next few months, but also feeling calm and confident. She visited her family in India in April. She told her father about the rollout excitement, which happened in batches the following month. Over Labor Day weekend, Banerjee was visiting her son in Texas as the algorithm blocked or redirected 5,000 potential party bookings. But no matter how quickly the AI models learn, Banerjee and her team will need to continue to monitor MARKETS party-inclined users figure outways around the barriers.  and change the systems as VIDEO WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 10/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years “The interesting part about the world of trust and safety is that it never stays static,” Banerjee LIVESTREAM SIGN IN said. “As soon as you build a defense, some of these bad actors out there who are potentially trying to buck the system and throw a party, they will get smarter and they’ll try to do something different.”  VIDEO 04:41 Airbnb’s unwelcome guests Follow your favorite stocks CREATE FREE ACCOUNT In this article ABNB +0.15 (+0.13%) TRENDING NOW A recession is coming in the U.S., and ‘a few rate cuts’ won’t prevent it, says strategist 1 85-year-old mom who co-signed daughter’s student loan years ago now fears lender may take her house 2  MARKETS  VIDEO  WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 11/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years Fed Chair Powell indicates interest rate cuts ahead: ‘The time has come for policy to LIVESTREAM SIGN IN adjust’ 3 Dow closes more than 450 points higher as Powell signals Fed rate cuts are coming 4 42-year-old sold his startup for $1.3B—he started by buying a $17,500 camera he couldn’t afford 5 Subscribe to CNBC PRO Subscribe to Investing Club Licensing & Reprints CNBC Councils Select Personal Finance CNBC on Peacock Join the CNBC Panel Supply Chain Values Select Shopping Closed Captioning Digital Products News Releases Internships Corrections About CNBC Ad Choices Site Map Podcasts Careers Help Contact        News Tips Got a confidential news tip? We want to hear from you. GET IN TOUCH Advertise With Us PLEASE CONTACT US  MARKETS  VIDEO  WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 12/13 8/23/24, 7:21 PM Airbnb party preventer Naba Banerjee reduced parties 55% in two years  CNBC Newsletters Sign up for free newsletters and get more CNBC delivered to your inbox LIVESTREAM SIGN IN SIGN UP NOW Get this delivered to your inbox, and more info about our products and services. Privacy Policy Your Privacy Choices CA Notice Terms of Service © 2024 CNBC LLC. All Rights Reserved. A Division of NBCUniversal Data is a real-time snapshot *Data is delayed at least 15 minutes. Global Business and Financial News, Stock Quotes, and Market Data and Analysis. Market Data Terms of Use and Disclaimers Data also provided by  MARKETS  VIDEO  WATC H L I S T MENU https://www.cnbc.com/2023/09/19/airbnb-party-preventer-naba-banerjee-reduced-parties-55percent-in-two-years.html 13/13