Statistical Modeling: Finding a Winning System

Brian’s picks went 0-2 on Sunday, making his record so far in the NCAA tournament a combined 2-7. So what does Brian’s 2-7 win-loss record tell us about his handicapping system? Very little. It’s not possible to determine the long term advantage of Brian’s system based on a tiny sample of 9 picks. Evaluating a handicapping system based on 9 picks is like trying to assess if a coin is biased after only 9 flips. Even in games or ventures that present the opportunity for skilled and knowledgeable players/investors to gain an edge, natural variability has much to do with what happens in the short and mid-term. Given a large enough sample size however, it is possible to determine if a system or hypothesis is valid.

Blackjack has an inherent advantage over sports handicapping and financial trading in that the system for winning at blackjack is black and white and has been proven for more than 40 years. Back in 1959, Professor Ed Thorp had a hypothesis that blackjack could be beat. Thorp formulated and tested his system using the supercomputers at MIT. Computer simulations played out millions of blackjack hands, and confirmed that basic strategy is the optimal playing strategy and that card counting yields a long term advantage for players. The validity and effectiveness of card counting has been irrefutably proven; when most blackjack players fall short it is due to a lack of knowledge or a lack in skill in applying the system.

For those who wish to make sports handicapping or financial trading a profession, a model must be built and then tested to determine if it will make money. The keys to building a successful model are developing a hypothesis, testing the hypothesis by crunching numbers using an ample and reliable database, defining a model, and back-testing the model by applying it to historical data to see if it’s profitable. The goal is to arrive at the predictors which forecast future outcomes such as final scores and market movements. Even if back-testing the model indicates profitability there is a possibility the model may no longer be as profitable or viable at all. Since the back-testing is based on past data, Vegas line makers may now take the predictive model into account or the stock market may have adjusted to the market bias. This is an issue that does not apply to blackjack since the variables (house rules and conditions) are very easy to account for when determining the value of a a particular game. In future posts we will explore in more detail how to build predictive models.