Bookmaker Odds Evaluation (NBA 2018/19) PDF

So in the last session we started looking at bookmakers odds and evaluating their reliability. And we found that across the season they were actually a fairly reliable indicator of the ultimate outcomes in terms of win percentages or points scored for most leagues, but not necessarily all leagues. I...

So in the last session we started looking at bookmakers odds and evaluating their reliability. And we found that across the season they were actually a fairly reliable indicator of the ultimate outcomes in terms of win percentages or points scored for most leagues, but not necessarily all leagues. In fact, we found in the Indian Premier League cricket it was actually a fairly poor predictor. So what we're going to do in this session is to look in a bit more detail ways of evaluating the accuracy of bookmaker odds. And ultimately we're going to look at something called the briar school, which is in some ways one of the best ways of evaluating reliability. Okay, so we're going to look at first start looking at the NBA in the 2018/19 season, so we install the packages, load the data. And then again, we'll write down the win probability and we're always remembering that these are decimal odds, and so the win probability simply one divided by the decimal odds. And then we have to scale it for the over round. So this is a two outcome league win or lose and therefore we scale it by the some of the probabilities derived from the decimal odds for a win and a loss and that we do here, okay? Now another way of thinking about the reliability of the betting odds in terms of each individual outcome is to think about whether they predicted the correct result? In the sense that did the bookmakers make it more likely that the team would actually won would be the winner or less likely the team that actually won would be the winner? So in other words, were the bookmaker's odds probabilities over 0.5 for the actual winner? And we might think that if the bookmaker probability is over 0.5 then the bookmaker is some sense it's a success, they got the answer right. And if it was less than 0.5 less than 50% probability, then they got it wrong. So we can evaluate the bookmaker's odds on that basis. So first we create a dummy variable which has a value equal to 1 if the win probability is greater than 0.5 and zero. And so we just create that variable and the next line says, well, let's identify whether the odds were correct. And we say the odds were correct if the actual outcome, the win value was equal to the win prediction that we just generated. And then from that if that was true, you get a value of 1, if it was not true then you get a value of 0. And what we're interested in is the average success rate across the season. So remember this is all games played in the season, and you can see here we get a value for getting the odds correct of 60.67 or 67%. So what this means is that just over two thirds of the time, the betting odds were correct in the sense that the bookmaker said there was a more than 50% chance that the actual winner would be the winner in the game. Bookmakers odds vary according to how likely it is that the actual winner will win the game and teams can be more of a favorite or less of a favorite. And you might think that in those games where the bookmaker attributes a very high probability that the team will win, they might be even more likely to be accurate, and we can check that. So for example, let's take an artificial threshold of 0.6, 60%. How often will the bookmakers write when they gave a win probability of more than 0.6 to the outcome being the team would win the game. And so we calculate that here as NBA high prob, so if the probability is greater than 0.6, we'll give it a value of 1, and we'll give it a value of 0. And then we will calculate then based on that, we'll calculate the mean for the success rate of the bookmaker in setting the odds. So we run this now, you can see here we've created this value high prob and just picked on those games where that hype that the probability was greater than 0.6. So what was the success rate in games where the bookmaker placed the probability of more than 60% that the team would win. We just do the same calculation here and you can see here now the value is just under 75%. So in three quarters of the games where the bookmaker said the probability was above 60%, in three quarters of those games the bookmaker got it right. So you can you can see that there are going to be differences depending on the size of the probability attributed to the bookmaker to the outcome of a particular game. If the bookmaker's odds are close to 50% then the bookmakers probably going to get it wrong quite a lot. When the bookmaker's odds are closer to 100% the bookmaker is likely to get the outcome right almost all of the time. And there's a self test here where you could try different values here. So this gives you an idea of how the bookmaker's odds are working, but this is not a very satisfying measure overall of the success rate of the bookmaker in setting the probabilities. And that's where the Brier Scores comes in.