Rating System Challenge: Pomeroy Slips, ESPN Users Looking Sharp

After a crazy Sweet Sixteen in which yours truly picked one—yes, one—winner correctly, there's been a bit of a shake-up in the Rating System Challenge standings. Once in first place and in the 98th percentile of ESPN Tourney Challenge Brackets, Pomeroy has tumbled to seventh and 55th.

Pomeroy switched places with the ESPN National Bracket, which clearly indicates that most ESPN users didn't rely all that heavily on the ESPN Decision Tree tool (dead last).

Just as last week, FiveThirtyEight remains tied with Pure Chalk for second place, although their ranking among ESPN users has dropped six percentiles. This tells much of the story with these ratings systems. The best among among them ranks only in the 93rd percentile, and it quickly drops off from there.

Moreover, picking the best system through the Sweet Sixteen would have won you only 10 extra points over chalk. Not very impressive.

RNK	ENTRY	R64	R32	S16	E8	FF	NCG	CHAMPION	PPR	TOTAL	PCT
1	ESPN National Bracket	230	200	160	0	0	0	Ohio St	240	580	93.2
2	FiveThirtyEight	250	200	120	0	0	0	Ohio St	240	570	90.6
2	Pure Chalk	250	200	120	0	0	0	Ohio St	240	570	90.6
4	Vegas	210	200	120	0	0	0	Ohio St	240	530	72.5
5	Nolan Power Index	230	200	80	0	0	0	Kansas	560	510	61.1
5	Sonny Moore	230	200	80	0	0	0	Ohio St	240	510	61.1
7	Pomeroy	260	200	40	0	0	0	Ohio St	240	500	55.0
8	Lunardi RPI	220	180	80	0	0	0	Kansas	560	480	42.7
8	NCAA Official RPI	220	180	80	0	0	0	Kansas	560	480	42.7
10	Jeff Sagarin	230	200	40	0	0	0	Ohio St	240	470	36.9
11	Preseason AP Poll	240	140	80	0	0	0	Duke	80	460	31.4
12	LRMC Bayesian	250	160	40	0	0	0	Ohio St	240	450	26.4
13	ESPN Decision Tree	230	160	40	0	0	0	Kansas	560	430	17.8

More analysis after the jump.

We should expect the Nolan Power Index make a run for first place in the final three rounds, as they're the only system in the top 50% of users that has their national champion pick intact. With three of the four #1 seeds out of the tournament, late-round chalkiness has resulted in a Final Four disintegration for our systems.

Final Four Picks	East	West	Southwest	Southeast
Vegas	OSU (1)	Duke (1)	Kansas (1)	Pitt (1)
ESPN National Bracket	OSU (1)	Duke (1)	Kansas (1)	Pitt (1)
Sonny Moore	OSU (1)	Duke (1)	Kansas (1)	Pitt (1)
FiveThirtyEight	OSU (1)	Duke (1)	Kansas (1)	Pitt (1)
Jeff Sagarin	OSU (1)	Duke (1)	Kansas (1)	Pitt (1)
Pomeroy	OSU (1)	Duke (1)	Kansas (1)	Pitt (1)
NCAA	OSU (1)	SDSU (2)	Kansas (1)	BYU (3)
ESPN Decision Tree	OSU (1)	Duke (1)	Kansas (1)	BYU (3)
Lunardi RPI	OSU (1)	Texas (4)	Kansas (1)	BYU (3)
Nolan Power Index	OSU (1)	SDSU (2)	Kansas (1)	BYU (3)
LRMC Bayesian	OSU (1)	Duke (1)	Kansas (1)	Belmont (13)

Image credit: Wikipedia