Pool C

[Dave 'd-mac' McHugh]

WRT to your interview on Hoopsville last night and this chat from a few days ago, it seems that the problem isn't that the SOS # is incorrect as calculated, but that the algorithm the NCAA now uses with the HAM(sums each team's opponents wins and losses and applies the multiplier to both columns (to come up with OWP and OOWP) rather than averaging the percentages of each component) doesn't achieve the purpose intended for home/away games as well as your recommendation would.
  I think it's important enough that the national RAC chairman should get a response from the NCAA justifying why their algorithm is preferable to yours; if it's not mathematically rigorous(e.g., somebody thought it was a good idea at the time), it should be easy enough to change before the next regional rankings. The NCAA should be responsive to their clients.
  Thanks for your work in this area!

I don't know what the official stats guys said but at the time I believe we thought they were summing everything to avoid giving equal credit to games against teams with fewer D3 games on the schedule.  They were worried a 10-0 team would look as good as a 24-0 team.

To be clear on something ronk said (in bold) ... there was a time the math was being done as Matt and others expected it to work. That's how Matt was able to hone in on the math to produce his SOS numbers. But a few years ago, it suddenly changed. So the HAM was already in place and being utilized ... and getting what was expected from the SOS. Something has changed, we don't know why, but that has to be solved ... it isn't that the HAM isn't getting what we want from it. We know the HAM works as expected when the math is done how it was originally done.

[Ryan Scott (Hoops Fan)]

WRT to your interview on Hoopsville last night and this chat from a few days ago, it seems that the problem isn't that the SOS # is incorrect as calculated, but that the algorithm the NCAA now uses with the HAM(sums each team's opponents wins and losses and applies the multiplier to both columns (to come up with OWP and OOWP) rather than averaging the percentages of each component) doesn't achieve the purpose intended for home/away games as well as your recommendation would.
  I think it's important enough that the national RAC chairman should get a response from the NCAA justifying why their algorithm is preferable to yours; if it's not mathematically rigorous(e.g., somebody thought it was a good idea at the time), it should be easy enough to change before the next regional rankings. The NCAA should be responsive to their clients.
  Thanks for your work in this area!

I don't know what the official stats guys said but at the time I believe we thought they were summing everything to avoid giving equal credit to games against teams with fewer D3 games on the schedule.  They were worried a 10-0 team would look as good as a 24-0 team.

To be clear on something ronk said (in bold) ... there was a time the math was being done as Matt and others expected it to work. That's how Matt was able to hone in on the math to produce his SOS numbers. But a few years ago, it suddenly changed. So the HAM was already in place and being utilized ... and getting what was expected from the SOS. Something has changed, we don't know why, but that has to be solved ... it isn't that the HAM isn't getting what we want from it. We know the HAM works as expected when the math is done how it was originally done.

I'm not sure that's true.  We need to double check, but I think it was when the Home/Away multiplier went in that the math messed up.  We had H/A for a while with the old system, but when they went to SOS the first few years were without the multiplier and, I believe, the math problems came when they put it in.  I think the NCAA has been applying it the same way the whole time, it just took a little while for Matt to figure out what they were doing.

[KnightSlappy]

I believe Dave is correct although I could not recall it during our Hoopsville segment. Here's the d3blog about it from 2013. I believe the HAM debuted in 2011 (with 1.4/0.6 weights)

NCAA changed the SOS calculation
https://www.d3blogs.com/d3hoops/2013/02/11/mens-strength-of-schedule-calculations-changed/
http://tomaroonandgold.blogspot.com/2013/02/definitive-proof-that-ncaas-new-sos.html

My original take on the HAM (1.4/0.6)
http://tomaroonandgold.blogspot.com/2011/02/ncaa-doesnt-earn-any-trust-points-with.html

NCAA changed HAM to 1.25/0.75
http://tomaroonandgold.blogspot.com/2011/12/ncaa-earns-some-points-back-with.html#more

I used to write a fair bit. Maybe I should do that again?

[Ryan Scott (Hoops Fan)]

So it was when they changed the multiplier that things got wonky.  I gotcha.

I was just looking through the Pre-Championship Manual today; it's funny how all the math for every other calculation is spelled out plainly, but the multipliers are just described, with no examples.

[AO]

So it was when they changed the multiplier that things got wonky.  I gotcha.

I was just looking through the Pre-Championship Manual today; it's funny how all the math for every other calculation is spelled out plainly, but the multipliers are just described, with no examples.

The funny part was how they changed the calc without changing the manual.

2013 manual with the calc that they weren't actually using:


2020 manual:

Even before applying the multiplier to the new calc we can see a difference because the new calc gives a larger weight to the game against the team that played 28 games compared to the games against the teams with only 24 D3 games.  The new calc has the OWP at .561 while doing it the old way would make it .5518.

[thebear]

I believe Dave is correct although I could not recall it during our Hoopsville segment. Here's the d3blog about it from 2013. I believe the HAM debuted in 2011 (with 1.4/0.6 weights)

NCAA changed the SOS calculation
https://www.d3blogs.com/d3hoops/2013/02/11/mens-strength-of-schedule-calculations-changed/
http://tomaroonandgold.blogspot.com/2013/02/definitive-proof-that-ncaas-new-sos.html

My original take on the HAM (1.4/0.6)
http://tomaroonandgold.blogspot.com/2011/02/ncaa-doesnt-earn-any-trust-points-with.html

To Matt's point, I would be interested in what the real Home/Away W-L is for the entire NCAA D3.  I am tracking just one conference, and with 2 reg season days left, I am seeing a 43-38 Home/Away win split, which would suggest more like the 1.15 : .85 that Matt is suggesting.

NCAA changed HAM to 1.25/0.75
http://tomaroonandgold.blogspot.com/2011/12/ncaa-earns-some-points-back-with.html#more

I used to write a fair bit. Maybe I should do that again?

[KnightSlappy]

I have D3 home teams as 2493-1880 (.570) this season (when playing another D3 team, leaving out neutral site games) which suggests something like 1.15 is more appropriate than 1.25.

[Dave 'd-mac' McHugh]

The multiplier did go into effect in 2011 (I was able to confirm that looking at other manuals) ... and the math went sideways around 2013.

So at one point, they were doing the math the way KnightSlappy describes because I also know he talked to them about it to narrow down his math. Then it changed and we never got an explanation for that change.

[Dave 'd-mac' McHugh]

So it was when they changed the multiplier that things got wonky.  I gotcha.

I was just looking through the Pre-Championship Manual today; it's funny how all the math for every other calculation is spelled out plainly, but the multipliers are just described, with no examples.

I went through EVERY manual last night in DIII ... time I will never get back in my life ... they don't spell it out anywhere. Even in M/W Ice Hockey where they use their own RPI system with a weight (1.2/0.8) they don't spell it out.

It is also sad to see how many manuals are so badly put together. I suggested YEARS ago to several liaisons they should make sure every manual looked the same where parts are identical ... clearly no one has tried to even consider it outside of maybe common sports.

For those curious: I narrowed down that men's basketball and men's lacrosse are essentially the only two sports using the HAM with the current SOS (ice hockey uses a HAM, but has different "math" as it where since they have it incorporated with an RPI). Men's soccer used it from 2011 to 2016. Women's soccer only used it in 2016 (based on what their manual say for both sports). Other sports I thought had used it like women's volleyball, but there is no mention of whether they did or didn't use it in the past.

Maybe it's time to get rid of the multiplier. That said, I know it was used to kick coaches out of their gyms (how many examples can we give) and to not allow programs to get an unfair bump because they decided to play everyone at home. There are still programs who host multiple tournaments set up like this ... so I can see an argument for keeping it. I would say the same argument is in place for men's lacrosse where far too many big programs bully teams who want to play them into coming to their place.

[ronk]

 So, why is it that the NCAA doesn't respond to their constituency in this regard(how the SOS is computed and why this method as opposed to some other algorithm)?

[KnightSlappy]

For those curious: I narrowed down that men's basketball and men's lacrosse are essentially the only two sports using the HAM with the current SOS (ice hockey uses a HAM, but has different "math" as it where since they have it incorporated with an RPI). Men's soccer used it from 2011 to 2016. Women's soccer only used it in 2016 (based on what their manual say for both sports). Other sports I thought had used it like women's volleyball, but there is no mention of whether they did or didn't use it in the past.

I think the multiplier is very good. It just needs to be implemented correctly. The problem with the multiplier is arithmetic, not principle.

[Dave 'd-mac' McHugh]

So, why is it that the NCAA doesn't respond to their constituency in this regard(how the SOS is computed and why this method as opposed to some other algorithm)?

You are assuming someone has asked them ... they respond to members, but we don't know if any membership has raised the issue officially.

[Dave 'd-mac' McHugh]

For those curious: I narrowed down that men's basketball and men's lacrosse are essentially the only two sports using the HAM with the current SOS (ice hockey uses a HAM, but has different "math" as it where since they have it incorporated with an RPI). Men's soccer used it from 2011 to 2016. Women's soccer only used it in 2016 (based on what their manual say for both sports). Other sports I thought had used it like women's volleyball, but there is no mention of whether they did or didn't use it in the past.

I think the multiplier is very good. It just needs to be implemented correctly. The problem with the multiplier is arithmetic, not principle.

I was playing devil's advocate to some degree ... but also considering so few committees have it as a tool I was wondering if it was something that was needed now. I don't really have an answer ... just curious.

[Ryan Scott (Hoops Fan)]

So, why is it that the NCAA doesn't respond to their constituency in this regard(how the SOS is computed and why this method as opposed to some other algorithm)?

You are assuming someone has asked them ... they respond to members, but we don't know if any membership has raised the issue officially.

The committee has to have at least inquired, right?  They seem very concerned with doing their job to the best of their abilities.  If it's an issue of implementation over statue, shouldn't they have some leeway on how the math is done?

[Dave 'd-mac' McHugh]

So, why is it that the NCAA doesn't respond to their constituency in this regard(how the SOS is computed and why this method as opposed to some other algorithm)?

You are assuming someone has asked them ... they respond to members, but we don't know if any membership has raised the issue officially.

The committee has to have at least inquired, right?  They seem very concerned with doing their job to the best of their abilities.  If it's an issue of implementation over statue, shouldn't they have some leeway on how the math is done?

I can't say if they have or not. Not my place and I don't want to assume anything. They may have, but I can't confirm it, so I am not going to assume it.