D3 Basketball Ranking System

augie_superfan · November 14, 2011, 11:41:22 PM

So, the season is finally upon us. Over the summer, I developed a basketball ranking system similar to Ken Massey's rankings. I've made some adjustments to his ideas. The biggest difference is that while Massey takes into account all games across all divisions, my system only includes games between Division III teams. By replaying last year's season day-by-day, my system predicted the winner of roughly 74% of the games.

I hope to have at least weekly ratings, which I will link to from the boards. I have a basic website setup. Right now, all that is there is an explanation of the system, last year's final top 50, and to hopefully get some excitement for the season upcoming season, I've posted estimated margins-of-victory for tomorrow's games (Note: these are estimated basically by using last year's final rankings, thus they will be much less accurate until a sufficient number of games are played).

This was just a little fun experiment to better understand ranking systems but I figured the results from last year were pretty good so why not share it with others that might be interested. Let me know if you have questions...below is the link:

https://sites.google.com/site/d3basketballindex/

KnightSlappy · November 14, 2011, 11:52:12 PM

Brilliant. Absolutely love it!

smedindy · November 15, 2011, 10:25:28 AM

My only sad is that no Presque Isle ratings.

Otherwise

Ralph Turner · November 15, 2011, 01:41:21 PM

I think that you underrepresent the ASC and schools in Texas. We have few chances to play teams from outside the area. Therefore the interaction necessary to predict the strength of a team is missing. I believe that ASC Champ and ASC-West Division winner McMurry, ASC-East champ UT-Dallas and Sweet 16 (Pool C) UMHB were all Top 50 teams last season.

To accomodate Centenary, the conference will play 8 crossover games and 14 district games.

That leaves only 3 non-conference games, and few dates to play those opponents. The one way that you can help the isolated teams is to include non-D-III scores.

Thanks for the effort.

sac · November 15, 2011, 01:48:34 PM

The MIAC schools can only play 4 non-conference games each. They have 6 teams ranked on augie_superfan's excellent ranking.

Ralph Turner · November 15, 2011, 01:51:40 PM

Quote from: sac on November 15, 2011, 01:48:34 PM
The MIAC schools can only play 4 non-conference games each. They have 5 teams ranked on augie_superfan's excellent ranking.

Thanks for the comment.

Yes, but there are several other schools around the MIAC to which the computer rankings can link. Their near-by universe is much more "robust", (I hope that that term is representative of the statistical modeling.)

smedindy · November 15, 2011, 04:42:58 PM

I think the effort is to make this a closed universe, as a more open universe makes it difficult to put the value proposition on outside-D3 teams. If you want this to be a robust ratings system, you need to filter out 'noise' and arbitrary saying this D2 team has this value or this USCAA team has that value makes it harder to really peg a true value.

Even with a limited slate of non-conference games there should be enough of a cross-over to connect the ASC teams to the rest of the universe. They're not the NESCAC in football!

Ralph Turner · November 15, 2011, 05:59:31 PM

Thanks, smed. My concern about the validity of his rankings is that UMHB finished #16 at the end of the season on the "big poll". UT-D received votes last year (RV #33) after losing in the ASC-Semi-finals to McMurry and then losing the first round game as a Pool C bid to UMHB. The statistical model has neither in the Top 50. Using last year's results in a D-III universe, then you are likely to understate the quality of play in the ASC.

After removing the playing dates that the ASC mandates for crossover play, that only leaves about 1 week at the start of the season and Christmas break. That makes it difficult to find D-III opponents. I find 17 games against 5 SCAC-West schools. Hardin-Simmons goes to Las Vegas for 2 games and UW-Stout comes to Belton (UMHB) in December. Other than that there is very little cross over.

I can accept that, but all that use the tool may not be aware of that problem.

augie_superfan · November 15, 2011, 07:13:12 PM

Quote from: Ralph Turner on November 15, 2011, 05:59:31 PM
Thanks, smed. My concern about the validity of his rankings is that UMHB finished #16 at the end of the season on the "big poll". UT-D received votes last year (RV #33) after losing in the ASC-Semi-finals to McMurry and then losing the first round game as a Pool C bid to UMHB. The statistical model has neither in the Top 50. Using last year's results in a D-III universe, then you are likely to understate the quality of play in the ASC.

After removing the playing dates that the ASC mandates for crossover play, that only leaves about 1 week at the start of the season and Christmas break. That makes it difficult to find D-III opponents. I find 17 games against 5 SCAC-West schools. Hardin-Simmons goes to Las Vegas for 2 games and UW-Stout comes to Belton (UMHB) in December. Other than that there is very little cross over.

I can accept that, but all that use the tool may not be aware of that problem.

Ralph,

I completely agree with you that the lack of out-of-conference games for the ASC and other isolated schools can lead to issues in the rankings. However, the issue is not that it automatically misrepresents the league on the lower side. It would just mean there is less information to come to a conclusion. All games have the same significance, more games for one conference just gives more information to better fit the data, not actually any advantage to that conference.

Where I do agree with you is when you look at the excluded games in my system. I agree that more information is probably better but it's hard to tell without building two systems and comparing them. Unfortunately I dont have the time to track all basketball teams. When you look at specific teams you can pick out where there may be issues. As you mentioned, McMurry was a team that probably should've been rated a bit higher if all games were included. I see 2 games, a close loss to Southern Methodist and a blowout win against Barclay. By looking at Massey's rating from last year I see that both of these were what I would call as "over-performances"...meaning games where they beat the expectation by a lot (actually major over-performances). By leaving these out, my system then underestimated their true ranking with all info included. If I scale my rankings with this info than it would raise their ranking from roughly #63 to #48. This actually seems to be an extreme case so I'm glad you brought it to my attention.

If I can get some time, I will try to take a look at some of the other ASC teams and see if games that were excluded would've been considered over or under performances.

I had given some thought to not tracking every team but having "dummy" teams like "D1 Top 25", "D1 25-100", "D2 1-50", etc. to account for the game and the opponents strength. It would only add the tracking of maybe 30 "teams" instead of thousands more....maybe next year!

augie_superfan · November 15, 2011, 07:20:51 PM

Quote from: sac on November 15, 2011, 01:48:34 PM
The MIAC schools can only play 4 non-conference games each. They have 6 teams ranked on augie_superfan's excellent ranking.

Actually one of the things that got me interested in these rankings systems last year was the MIAC's dominance in the rankings. I couldn't believe that the Massey ratings were right in having teams like Carleton, Hamline, and Gustavus Adolphus up in the Top 10-25. Mine rated them a bit lower but not as much as I would've thought.

This is the fun of my system, the chance to talk about what you agree and disagree with, all the opinions give me more things to look at and hopefully for ways to make it better....thanks for the feedback.

mailsy · November 15, 2011, 09:38:26 PM

+1 on this ranking. Very Cool.

You had scoring pretty dead on. Cabrini beat Haverford by 12.

augie_superfan · November 15, 2011, 10:12:45 PM

Quote from: mailsy on November 15, 2011, 09:38:26 PM
+1 on this ranking. Very Cool. You had scoring pretty dead on. Cabrini beat Haverford by 12.

Anything this early in the season is just plain luck...the system has no information to go by except last year's final rankings...thus it doesn't take into account the changes of team strength via graduation, new players, etc.

I'm sure there are plenty games tonight that are off by more than 20 points. Thanks for checking out the rating though...keep checking back.

Ralph Turner · November 17, 2011, 12:16:25 AM

Augie, I can understand the Barclay and SMU games for McMurry.

The more problematic question is why are UMHB (#16) and UT-Dallas (RV #33) not even found in the Top 50!

Thanks for the good work! +1!

augie_superfan · November 17, 2011, 08:24:31 AM

I think maybe I miscommunicated what the "Top 50" were that I posted. Those were the Top 50 in my system from the end of last year. I think you are taking the #16 and #33 from this year's preseason poll. It looks like at the end of last year, UMHB was ranked #20 and UT-D was ranked #44. In my final ranking, they were #68 and #57 respectively. If the full McMurry data was in the system, I estimate it would've moved UMHB up roughly 6 spots due to them playing 4 times last season.

If you go and look at the Week 13 poll, they were ranked, UMHB(#44) and UT-D(#33). So UMHB made a major move in the polls due to 3 games whereas those 3 games affect my system little because they slightly over-performed twice and then under-performed at Augustana. Now, in the eyes of a human voter, these were 3 big games and thus why there was a large movement in their rank. However, for my system, these games could've been played the first week of the season or the last and my system didn't care. Now, there are some other ranking systems that feel differently than this and either weight tourney games more or recent games more, etc . So, I think if trying to compare these apples and oranges, we could actually also compare the numbers of UMHB(#44 vs. roughly #62) and UT-D(#33 vs. #57).

But this is precisely why you can't compare these rankings to the human polls because they are based off different "inputs" in a way. Also, other human voters probably "weight" games differently in their mind. Is a human poll supposed to take into account every game of the season with the same weight or should a team get more credit for winning in the tournament? That's a question that I don't think there is a right or wrong answer to.

Thanks for the comments

smedindy · November 17, 2011, 09:36:44 AM

Humans vote emotionally, and I think they weight the more current games highly as they vote. The beauty of these ratings system is that they take the results as a whole. Of course, they can't filter out injuries, travel, and other issues. But you need both the emotion and the cold hard facts of life.