Top 25 talk

[Pat Coleman]

Not sure that makes most of the D-III rankings useless. Were we expecting Calvin to beat Ferris State? If we think Calvin is better than Ferris State in real life but lost because it was an exhibition, then yes, that would invalidate the ranking. But I suspect Ferris State was going to be better than Calvin in the rankings regardless of whether that game counted or not.

[John Gleich]

I agree that there does seem to be some definite bias towards the WIAC, which calls into question the validity of the entire ratings.  I think that is shown when you look at the conference ratings and then look at the conference winning percentages.  I know that strength of schedule factors in... but that takes the rating into account... which, again, seems to have a WIAC skew.  Thus, 7-6 River Falls is #28 and the lowest rated WIAC team, Superior, is #72.  The entire conference is rated above Franklin and Marshall (#81).  And, as such, this has an effect, like Pat said, on conferences that the WIAC plays.  For instance, the MIAC went 4-6 against the WIAC...  St. Johns beat Eau Claire and Oshkosh, St. Thomas and Augsburg beat River Falls.  Because of the higher rating of each WIAC team, each win against them "counts" for more... and thus the MIAC is the #2 ranked conference, even though their win% is 7th (not taking vs. in-region D-III... purely wins and losses).

Win% rank	Massey	Conference	Win%
1	(1)	WIAC	76.9%
2	(4)	UAA	72.2%
3	(10)	NJAC	68.9%
4	(5)	ODAC	68.5%
5	(9)	NESCAC	67.4%
6	(2)	MIAC	64.8%
7	(12)	MAC	63.6%
8	(3)	CCIW	62.5%
9	(22)	SCAC	60.9%
10	(23)	NEWMAC	59.2%
11	(17)	Empire 8	58.2%
12	(11)	SUNY AC	58.2%
13	(28)	Little East	56.7%
14	(21)	Capital	56.5%
15	(6)	IIAC	56.3%
16	(7)	OAC	53.7%
17	(30)	Liberty League	52.2%
18	(27)	Great South	52.1%
19	(16)	HCAC	51.3%
20	(38)	Great Northeast	50.0%
21	(39)	Allegheny Mtn	50.0%

It seems pretty obvious to me that something is amiss...  Now, again, this is purely win%, it doesn't take into account OWP or OOWP... but Massey doesn't seem to, really either (in the same way that the D-III selection committee would).

I think the most obvious skews here are the CCIW and the IIAC...  Both conferences are bolstered by games with the WIAC, seemingly...  I kind of calls into question which is the cause and which is the effect...  The IIAC and MIAC have matched up too, so that might bring the IIAC up a bit...  But then take a look at the NWC.  Oh wait, you can't... they're not even on the list.  Let me include a few more on the list, including the NWC and the SCIAC:

Win% rank	Massey	Conference	Win%
22	(31)	Landmark	48.7%
23	(44)	North Atlantic	44.6%
24	(32)	Centennial	44.3%
25	(26)	Northern Athletic	44.0%
26	(34)	Presidents' AC	44.0%
27	(29)	Southern Cal IAC	43.9%
28	(43)	New England CC	42.6%
29	(15)	Michigan IAA	42.5%
30	(8)	Northwest	42.3%
31	(18)	Midwest	41.2%

Wow... the SCIAC has a better win %... but they're 21 slots lower in Massey.  And even though the NWC played 10 games vs. the WIAC while the SCIAC played 0, the NWC only won 3 of those games (though one was against Whitewater, who likely was the previous #1... undefeated, win over Stevens Point).

Some results are curious though... the NathCon didn't get any WIAC boost even though they played 'em 9 times (1-9).  Much like the UMAC.  I don't know how the MWC and NathCon did against each other this year... but this is a switch from years past.  The MWC actually HAD been one of the top 10 conferences (in terms of winning percentage).  This year... not so much (1-5 vs. the WIAC).

What's interesting is that there doesn't seem to be much affecton playing bad conferences on the top rated ones...  The WIAC (9-1) and MIAC (14-1) obliterated the UMAC and played 35 of their combined 157 games against the D-III bottom feeder... but that didn't seem to drag either conference down at all.  Maybe these are the "projected results" so there's no drop... I dunno.

[Hugenerd]

One other factor that has not been discussed in depth on this board yet is that Massey takes into account all games.  That means every game, D1, D2, NAIA, etc. is taken into account (which is probably not the case for the d3hoops pollsters and definitely not the NCAA committee).  A lot of the conferences and teams that have "inflated" massey ratings are those that play a lot of games against solid out of division opponenets (west coast teams, MIAC teams, etc.).  A lot (obviously not all) of these non-division teams have high ratings compared to your average d3 team and therefore they provide a boost in the massey ratings that you would not get from just playing a slate of purely d3 schools.  I am not sure how significant this factor plays, but it seems like it is having an effect.

[magicman]

I dont think Massey has the time to check all the d3 ratings closely, I am sure if you email him errors you find he will update them, he has been responsive in the past.

hugenerd,
You are correct. I found an exhibition listing in Plattsburgh State's record and e-mailed him about it. He responded a few hours later that he would fix it. As with anything of this magnitude (The Massey Ratings) there's bound to be a few glitches.

[John Gleich]

One other factor that has not been discussed in depth on this board yet is that Massey takes into account all games.  That means every game, D1, D2, NAIA, etc. is taken into account (which is probably not the case for the d3hoops pollsters and definitely not the NCAA committee).  A lot of the conferences and teams that have "inflated" massey ratings are those that play a lot of games against solid out of division opponenets (west coast teams, MIAC teams, etc.).  A lot (obviously not all) of these non-division teams have high ratings compared to your average d3 team and therefore they provide a boost in the massey ratings that you would not get from just playing a slate of purely d3 schools.  I am not sure how significant this factor plays, but it seems like it is having an effect.

I agree... but I wonder what the effect really is.  If you look at the "conference" ratings, the subcategories of NCAA-I, II, NAIA I, II, NCCAA, etc are there, but they don't have the ranking one would think they should have.  Per the D-III vs. DI, DII... etc thread, the rankings should be

D I
D II
NAIA I

And then NAIA II would likely be mixed in somewhere in the middle, with NCCAA towards the bottom.

Again, I'm not sure if the conference rankings are just pulling data from above, or if they're calculated separately... but the subgroups are not ranked in the seemingly obvious way.

It should be noted that there appears to be some kind of constant in affect.  NCAA-II and CCAA (A D-II conference in California for some reason listed here) have the same Power rating, offense, defense, etc.  Both have a parity of 1 (complete parity, top to bottom).  Similarly, two other "conferences" are listed as NCAA-III and NCAA.  These have all of the same numbers except for parity.  I'm wondering if these are factors that are used somewhere in the math... but it isn't obvious why they're listed here.

Great Plains AC (NAIA except for dual member Neb Wesleyan), NAIA-I and NAIA-II are all the same, with a parity of 1 as well.

The parity number is interesting... (again, I'm not sure how much of this data we can really use because of the obvious skew).  The two conferences with the highest (i.e. closest-to-one) parity number are the USA South and the UAA.  I'm wondering how that will shake out as the conference schedules play themselves out. 

[Titan Q]

One other factor that has not been discussed in depth on this board yet is that Massey takes into account all games.  That means every game, D1, D2, NAIA, etc. is taken into account (which is probably not the case for the d3hoops pollsters and definitely not the NCAA committee). 

Actually, I'm pretty sure all of the D3hoops.com Top 25 voters factor in all games.  That said, I'm not sure we always do a perfect job of recognizing a really good NAIA opponent (say, a team from the GSAC - Westmont, Concordia, Biola, etc)...I've seen us penalize a D3 team too severly for a loss to a strong NAIA.  Games vs NCAA I and II are easier to quickly identify.

But overall, the pollsters do count all games.  I try to keep an eye on the NAIA I and II ratings and watch for games vs these teams, just to gain a little perspective.

http://naia.cstv.com/sports/m-baskbl/spec-rel/121409aab.html

http://naia.cstv.com/sports/m-baskbl-div2/spec-rel/121409aab.html

[sac]

I dont think Massey has the time to check all the d3 ratings closely, I am sure if you email him errors you find he will update them, he has been responsive in the past.

Since I emailed him about the exact same error last year.......and it was never fixed, I have a different view of his responsiveness.

Calvin's 2009 exhibition with Ferris State still appears on their schedule.
http://www.masseyratings.com/team.php?t=1126&s=87798

Not sure that makes most of the D-III rankings useless. Were we expecting Calvin to beat Ferris State? If we think Calvin is better than Ferris State in real life but lost because it was an exhibition, then yes, that would invalidate the ranking. But I suspect Ferris State was going to be better than Calvin in the rankings regardless of whether that game counted or not.

Any team who has played Calvin or will play Calvin will have corrupted data, thereby rendering their ranking inaccurate.  The same would go for any other exhibition games counted for other teams and every opponent they face.

Massey has had this problem for more than a couple years now.

[Hugenerd]

I dont think Massey has the time to check all the d3 ratings closely, I am sure if you email him errors you find he will update them, he has been responsive in the past.

Since I emailed him about the exact same error last year.......and it was never fixed, I have a different view of his responsiveness.

Calvin's 2009 exhibition with Ferris State still appears on their schedule.
http://www.masseyratings.com/team.php?t=1126&s=87798

Maybe you didnt ask nicely enough?

Not sure that makes most of the D-III rankings useless. Were we expecting Calvin to beat Ferris State? If we think Calvin is better than Ferris State in real life but lost because it was an exhibition, then yes, that would invalidate the ranking. But I suspect Ferris State was going to be better than Calvin in the rankings regardless of whether that game counted or not.

Any team who has played Calvin or will play Calvin will have corrupted data, thereby rendering their ranking inaccurate.  The same would go for any other exhibition games counted for other teams and every opponent they face.

Massey has had this problem for more than a couple years now.

I think Pat's point is that the Ferris State result likely will have minimal effect on Calvin's ranking.  Since Ferris State is ranked higher and therefore the favorite (they are ranked 351 compared to Calvin's 865), it will likely not hurt their rating much at all.  In fact, it may help their SOS and actually increase their numbers.  It will obviously have some small effect, but I doubt without that one loss they would move very much in terms of their absolute ranking.

[David Collinge]

Data integrity is data integrity.  Suppose there were an election, and Candidate A wins with 55% of the vote.  It later turns out that one precinct admits that they didn't bother to check voter registration, and as a result allowed unregistered, invalid, and possibly even duplicate voters to cast ballots.  There were not enough voters in that precinct to sway the outcome of the election.  Is the election still valid?

[Hugenerd]

Data integrity is data integrity.  Suppose there were an election, and Candidate A wins with 55% of the vote.  It later turns out that one precinct admits that they didn't bother to check voter registration, and as a result allowed unregistered, invalid, and possibly even duplicate voters to cast ballots.  There were not enough voters in that precinct to sway the outcome of the election.  Is the election still valid?

I think you would be hard-pressed to persuade them to have a re-vote in the scenario you described.  Maybe they would throw out the votes from that precinct, but as long as it was not possible to sway the entire election, I think the election would still stand.  However, the situation with Calvin is not as simple as your election analogy.  In the election analogy, each vote is presumably independent; in the case of Calvin's Massey rating, however, all the numbers are dependent because they take into account opponents record, etc.

You may be thinking to yourself, "Well, he is just proving our point further," but even though the numbers will be slightly off and you could see small "ripples" of this error throughout other teams rankings as well, I dont think that error is large enough to invalidate the entire set of rankings and data.  Remember that this is one game out of thousands.  What if an AD reported a score wrong and a 6 point game was actually a 9 point game or there were other slight errors that were out of the control of Massey? These would all change the rankings slightly also. Remember that Massey essentially downloads results and schedules from other sources where large amounts of data are available in one place, it does not download from individual university sites.  He is not plugging these things in by hand and checking each of the tens of thousands of game each year.  In all likelihood, he downloaded the data from d3hoops.com or the NCAA which doesnt list this game as exhibition either (if you look at Calvin's page on d3hoops.com, the Ferris state game is not shown as an exhibition with the symbol designated in the key, a #; however, it is not counted in the total record at the top, which could be easy for a computer program to miss that is not programmed to do so; technically, d3hoops.com is at the same fault in terms of data integrity as Massey because they are not using their own designation for exhibition games).  I dont think the fault is any more Massey's than it is the source he downloaded from.  If you email him and he fixes it, then that is great.  If he doesnt, I still think the Massey ratings are a nice additional set of information to have.  They are the only national RPI index that rates all divisions, including d3, so maybe we should go a little easier on Mr. Massey.

[KnightSlappy]

Data integrity is data integrity.  Suppose there were an election, and Candidate A wins with 55% of the vote.  It later turns out that one precinct admits that they didn't bother to check voter registration, and as a result allowed unregistered, invalid, and possibly even duplicate voters to cast ballots.  There were not enough voters in that precinct to sway the outcome of the election.  Is the election still valid?

I think you would be hard-pressed to persuade them to have a re-vote in the scenario you described.  Maybe they would throw out the votes from that precinct, but as long as it was not possible to sway the entire election, I think the election would still stand.  However, the situation with Calvin is not as simple as your election analogy.  In the election analogy, each vote is presumably independent; in the case of Calvin's Massey rating, however, all the numbers are dependent because they take into account opponents record, etc.

You may be thinking to yourself, "Well, he is just proving our point further," but even though the numbers will be slightly off and you could see small "ripples" of this error throughout other teams rankings as well, I dont think that error is large enough to invalidate the entire set of rankings and data.  Remember that this is one game out of thousands.  What if an AD reported a score wrong and a 6 point game was actually a 9 point game or there were other slight errors that were out of the control of Massey? These would all change the rankings slightly also. Remember that Massey essentially downloads results and schedules from other sources where large amounts of data are available in one place, it does not download from individual university sites.  He is not plugging these things in by hand and checking each of the tens of thousands of game each year.  In all likelihood, he downloaded the data from d3hoops.com or the NCAA which doesnt list this game as exhibition either (if you look at Calvin's page on d3hoops.com, the Ferris state game is not shown as an exhibition with the symbol designated in the key, a #; however, it is not counted in the total record at the top, which could be easy for a computer program to miss that is not programmed to do so; technically, d3hoops.com is at the same fault in terms of data integrity as Massey because they are not using their own designation for exhibition games).  I dont think the fault is any more Massey's than it is the source he downloaded from.  If you email him and he fixes it, then that is great.  If he doesnt, I still think the Massey ratings are a nice additional set of information to have.  They are the only national RPI index that rates all divisions, including d3, so maybe we should go a little easier on Mr. Massey.

You're probably dead on here. I get all my data for the regional rankings (posted on the Pool C board) from d3hoops.com so when a game is incorrectly listed as in-region, my data suffers. I don't want (or have the time) to look through all of my d3 teams data every time, and I doubt Massey does so for ALL of college basketball. A game here or a game there does affect that specific team's data integrity, but for the entirety of division 3, it means little.

[John Gleich]

I don't necessarily think anyone is giving Mr. Massey a hard time...  I think it's laudible to try to project and fully rank teams... but I think there's a problem somewhere (more than just in the incorrect data from one or two games).

It appears (at least to me) there's too much weight being placed on the top teams.  It's having a trickle down effect and affecting the teams that are playing WIAC schools, and not effecting others.

But I might be wrong.  Knightslappy, do you have the ability to compile conference OWP/OOWP?  I realize that, because some teams have started conference play and other have not, they won't be perfect (in a perfect world, we'd be able to just look at the non-con schedules)... but that might shed more light on the results.

And yes, I realize I'm talking about something completely different than Sac... it's more along the lines of what Pat mentioned:

Massey ratings do seem to have this "proximity to the WIAC" thing going -- anyone connected to a WIAC team does really well.

I mean, I guess there's a chance that the WIAC's results are just breaking the system...  but I don't think so.  In DI, the Big 12 is the top conference... with a non-con record of 132-28 (win% of 82.5%).  As I look at the DI numbers, I don't see anything as agregious as the WIAC in terms of an entire conference being ranked in the top 72 teams... or for a 5-7 Stout team to be #57, directly ahead of two one-loss teams (Western Connecticutt and Birmingham Southern)... and 354 other teams, especially when they haven't been better than 58% of the teams they've played.  I'm not hating on Stout... they've played (and been beaten) by the three top 25 teams in the WIAC... who just happen to be 1, 2, and 6 in the Mratings, plus Whitworth, who's #5. That likely has lots to do with their current slot... but they've still been beaten by everybody in conference they've played.  Tonight's matchup with River Falls should tell us a lot about Stout... but RF is astronomically high too.

[Pat Coleman]

in the case of Calvin's Massey rating, however, all the numbers are dependent because they take into account opponents record, etc.

Does it take the record into account, or the rating? I maintain the rating probably isn't that much affected by counting a loss to Ferris State, since it's a result that fits with other Calvin results and would be expected anyway.

BTW, on the designation for exhibition games, I have long asked for Presto to fix that glitch. I can certainly ask again.

[Pat Coleman]

By the way, he doesn't download from us. He used to, but when we moved to Presto we were no longer in complete control of the data.

[Pat Coleman]

Any team who has played Calvin or will play Calvin will have corrupted data, thereby rendering their ranking inaccurate.  The same would go for any other exhibition games counted for other teams and every opponent they face.

Massey has had this problem for more than a couple years now.

Yeah, see, but what I'm saying is that adding an expected result -- Calvin losing to Ferris State -- isn't likely to have a big impact on the data. It will have some, yes, but on paper, Calvin was going to lose to Ferris State anyway, and counting that game doesn't change that.