Rating Systems Challenge: A Victory for ESPN BPI, a Review and a Mea Culpa

This post is part of a series about the 2016 NCAA Tournament.

First things first: there was a serious error in all my previous posts. I was working off of old data for Nolan Power Index. In fact, I was working off last year's data. Funnily enough, last year's NPI ratings outperformed every one of this year's systems in the 2016 Rating Systems Challenge. Unfortunately, that's not how this works. As usual, the real NPI ratings finished poorly, dead last. This means the winner was actually ESPN's BPI. This also means that not a single system this year picked the correct National Champion.

No system picked Villanova to win it all, but at least BPI, Vegas, FiveThirtyEight and LRMC outperformed chalk, with Lunardi RPI and Massey tying the default system. All of the aforementioned systems nailed more picks than chalk, as did the preseason and postseason polls, Sagarin, ESPN Computer, Ken Pomeroy, Sports Reference's net adjusted ratings, Team Rankings Predictive Power Ratings, Christopher Long's simulator, classic RPI, Sonny Moore and Nolan Power.

In the updated historical rankings above, we can see that the only systems that have reliably outperformed chalk in the past six tournaments were BPI, the ESPN Tournament Challenge National Bracket, FiveThirtyEight, Sagarin and Vegas; most systems actually severely underperform chalk.

Thank you all for sticking with me this year. Hang around for 2016 MLB coverage, if that's your jam. If not, see you next March!

Check my work:

• Constructed Brackets
• Links to Data
• AP Postseason Poll
• AP Preseason Poll
• Christopher Long
• ESPN BPI
• ESPN National Bracket
• FiveThirtyEight
• LRMC Bayesian
• Lunardi RPI
• Massey
• NCAA RPI
• Nolan Power Index
• Pomeroy
• Pure Chalk
• Sagarin
• Sonny Moore
• Sports Reference
• Team Rankings Predictive Power Rankings
• USA Today Postseason Poll
• USA Today Preseason Poll