NCAA Rule Change on Tournament Selection Criteria

[Hopkins92]

What's the rush? Is it mainly so they can get this slammed into being, knowing the coaches are going to gripe about it? I don't understand why they couldn't put it into effect but move the effective date out 2 seasons. They still get what they want but they don't (potentially) cost a coach his/her job by allowing them to adjust their scheduling accordingly.

Just weird to me. The rule is going to be a pretty dramatic shift for those schools that promote "competitive program, aiming to play in November" to their recruits. For some conference middle-dwellers, this is going to be a really big deal.

[Newenglander]

For fear of making this NESCAC-Centric (I can hear it already  ) to make it completely formulaic is a tough sell indeed. I think someone commented it may keep that 3rd or 4th NESCAC team from getting a bid but those 3rd/4th teams have routinely demonstrated they can make a run in the tournament while running a regular season gauntlet (I know its not the only conference like that).....theres always subjectivity but just make it the best teams get in!

[jknezek]

Best is always subjective. This is an attempt to make "best" according to a relatively transparent formula.

It won't work, people will still complain and argue. Schedules will get weaker to suit the metric.

In a decade they will change it again. There is no right answer. There are just different ways of getting answers people will argue about.

[stlawus]

The more I read about this the more I'm fine with this being implemented, at least for a few years.  Seeing a lot of the basketball folks mad because it means all the teams and leagues that get the most attention every year will likely get the short end of the stick.  Same for leagues in soccer like the NESCAC.  You've had years and years of benefiting from the old system, time for other leagues and teams to get a turn. 

[Kuiper]

I'm a big college hockey fan and I will say the transparency of that process is really refreshing. (With that said, there are tons of issues with the actual tournament, but that's so off topic I wont bore everyone with it.)

I know that it's being stated that soccer would use a different Pairwise formula, but I will say, in college hockey, the better/bigger conferences gobble up more spots and the lesser conferences almost always just get their AQ in. So, Hockey East and the Big Ten get 3 and 4 teams in, and Denver's (NCHC) conference usually gets 2 or 3... In a 16 team field, you're talking about a lot of non-AQs going fast.)

Just going by what I'm hearing about the system, the DIII proposal is similar to hockey in the sense that hockey uses Pairwise and not a committee.  So, they both have the benefits of transparency and fewer chances for selections to vary based on human decisions about the importance of certain factors etc.  The difference is that the proposal for all DIII sports will use a feature to address one particular problem - schools getting dinged by wins against lower-ranked teams because it hurts their SoS - that I guess is not part of the hockey formula.

[mngopher]

I think the parity and $$ are legitimate factors. Also think it is just so tough for a committee or an algorithm to do a legitimate comparison with so many teams so spread out. There is no perfect way to do this. Pairwise works in D1 hockey with about 65 teams, but D3 soccer is a much different animal. SOS has become a driving factor for selection, in my opinion a bit too much. Can't remember the teams, but in the last few years there have been a couple teams winning only like half their games and still getting an at-large bid. Curious to see what this looks like in practice.

[Kuiper]

Bianco is continuing his fight against the proposal, at least the rush to implement it for this year.

https://x.com/BMBianco/status/1774810791616581779?s=20

👋 it's me again re: NPI/Pairwise

@NCAADIII
 Champs Committee- by your own admission, you're building the plane while it's flying. Membership hasn't seen the algorithm. A couple of questions:

1) What are the benefits to '24-25 implementation?
2) LET US SEE THE DATA!

Although delay is the usual tactic of opponents of change, it's a reasonable question to ask what's the benefit of moving in 24-25 rather than 25-26.  My guess is the Committee would say that if the concern is about lack of transparency at the time of selection (Bianco's concern is about lack of transparency at the time of scheduling, which is different) and biased selections by humans, then delay just allows more concerns about transparency and bias (assuming you think this system reduces those concerns).  Moreover, there's probably no perfect time to implement something like this because scheduling of non-conference games often is done several years in advance through agreements for return trips. 

[Newenglander]

May be over thinking but does it drive D3 (see it factoring less for D1) coaches out of the more competitive conferences and into positions they feel more favorable to make tournament play?

[Kuiper]

Ohio Northern has posted its 2024 schedule and I think it is a useful example of both the fact that the rushed implementation will disadvantage some schools, while at the same time illustrating that it's not that easy to solve that issue whenever it is implemented.

ONU clearly has a challenging schedule.  It plays Kenyon, Calvin, Ohio Wesleyan, Colorado College, and UT Dallas, in August/September.  Plus, it plays John Carroll in November right before the OAC tournament.  All six of those teams were in the NCAA tournament last year and only UT Dallas did not advance beyond the first round.  ONU lost to Colorado College in 2OT in the third round.

In theory, ONU would not have scheduled such tough teams if it wasn't going to help them as much as more wins.  They might still win the OAC and the AQ, making it irrelevant, but they might qualify even if they come in 2nd or 3rd because the OAC probably supplies just enough strong opponents in Otterbein, Mount Union, etc to give them a good enough strength-of-schedule number and they could have a good W-L if they just replaced some of their more difficult non-conference opponents with easier ones.

One obvious way to do so is ditching the costly and time-consuming trip to Colorado Springs to play two tough opponents at altitude early in the season.  I expect some people anticipate there will be fewer trips of this nature with the new rules.  The problem is that Colorado College traveled to Ada, OH last fall and ONU is probably contractually obligated to make a return trip.  Colorado College also traveled to Dallas to play UT Dallas last fall and UTD is probably contractually obligated to make the return trip to Colorado Springs this year.  Even if Colorado College wanted to avoid the difficult games too because of the rule change, they've already shelled out a lot of money last year and it's hard to get any home games in Colorado Springs given their location.  They need these games.

The other difficult non-conference games are more local, so travel isn't as much of an issue, but most of them are still home and away obligations.  ONU hosted Kenyon last year and Kenyon is counting on them to come to Gambier this year.  ONU visited OWU last year and is counting on them to come back. Even Rose-Hulman is a return on a home and away deal and ONU lost that game.  So, it's not an easy game.

You might argue that this would all change if they delayed the change until Fall 2025 when those obligations are concluded, but that doesn't really track.  ONU has scheduled new opponents that are likely to be challenging this year and/or next, such as Wash U and Hope, and those are probably home-and-away games too given the long distances between ONU and each of those schools.  Moreover, if it were still under the old rules this year, ONU would want challenging games this year and likely would be required by its AD to negotiate home-and-away deals for budgetary reasons.

It's hard to imagine a scenario where there wouldn't be a transition period of non-ideal schedules because of the push to meet the needs of the current rule in one year and the new rule in the second year and the inevitability of home-and-away deals that cross over.  You might solve that with a gradual implementation rule over several years, but it would have to be awfully complicated since schools are differently situated with respect to how many open non-conference games they have in any given year.  It's unlikely you could construct a transition rule that wouldn't disadvantage as many schools as it helps.

And ONU is in a conference that is a probably a little easier this year than last year because of John Carroll's departure.  They might like their chances of winning all but 1-2 conference games. In other conferences, there's not much a coach can do because all of their conference games are pretty challenging.

The team that might be hurt the most under the new rules isn't ONU, which will eventually be able to find local teams that are a bit easier (although it's not clear why those teams won't also want to avoid ONU for the same reason), but a school like Colorado College.  The only way they can get non-conference teams to come to Colorado Springs is to provide SoS benefits of playing them and a quality opponent in a round robin.  But, Colorado College is way too unique to plan around.

[mngopher]

ONU clearly has a challenging schedule.  It plays Kenyon, Calvin, Ohio Wesleyan, Colorado College, and UT Dallas, in August/September.  Plus, it plays John Carroll in November right before the OAC tournament.  All six of those teams were in the NCAA tournament last year and only UT Dallas did not advance beyond the first round.  ONU lost to Colorado College in 2OT in the third round.

I'm wondering if some of what you are saying here is another reason why coaches from lower leagues/non-perennial power teams could be pushing this. Right now there is pretty much 0 incentive for a team with hopes of making the tournament and beyond to play anything but other powerhouse teams in non-conference play. Maybe one or two, but not many to go around. SOS drives everything, and is arguably more important than winning to the selection committee. Because of this it has become somewhat of a good ole boys club in terms of non-conf scheduling. Can't break in unless you are a regular NCAA tourney team, but it's hard to become a regular tournament team if you can't get any of them to play you. Again, I see both sides of it. You want to match your team up against the best, I get that.

[Kuiper]

ONU clearly has a challenging schedule.  It plays Kenyon, Calvin, Ohio Wesleyan, Colorado College, and UT Dallas, in August/September.  Plus, it plays John Carroll in November right before the OAC tournament.  All six of those teams were in the NCAA tournament last year and only UT Dallas did not advance beyond the first round.  ONU lost to Colorado College in 2OT in the third round.

I'm wondering if some of what you are saying here is another reason why coaches from lower leagues/non-perennial power teams could be pushing this. Right now there is pretty much 0 incentive for a team with hopes of making the tournament and beyond to play anything but other powerhouse teams in non-conference play. Maybe one or two, but not many to go around. SOS drives everything, and is arguably more important than winning to the selection committee. Because of this it has become somewhat of a good ole boys club in terms of non-conf scheduling. Can't break in unless you are a regular NCAA tourney team, but it's hard to become a regular tournament team if you can't get any of them to play you. Again, I see both sides of it. You want to match your team up against the best, I get that.

Based on what Bianco has been tweeting, I'm not sure if coaches have had any input to push this one way or another, but I could see that being a complaint of non-traditional powers.  If there was a poster child in men's soccer for this reform it would be the 2022 Western Connecticut team that was undefeated (20-0-2), but didn't get the AQ because it lost in PKs to UMass Boston in the finals of the Little East Conference, a conference whose members likely did no favors for Western Conn's SoS.  Williams, by contrast, which went 10-2-11 in the NESCAC was selected over Western Connecticut for a Pool C bid after losing in the semifinals of the NESCAC tournament. Now, Williams justified their selection by advancing to the finals of the 2022 NCAA tournament before losing to Chicago, but there were plenty of people griping about how a team could go undefeated and still get passed over for a team that tied more games than it won, albeit in a tougher conference.  Western Conn had a good record in 2021 (17-6), but it wasn't a NCAA tournament team and it probably didn't help anyone's SoS to play them.  It wouldn't be worth it given that there was a risk that a strong team could lose to them.  In a SoS-based system, there's nothing worse than playing a lower team that beats you.  So, they probably felt hard done by the SoS-heavy selection criteria. 

To their credit, Wesleyan and Williams both added Western Conn to their 2023 schedule for the first time since at least 2011, if not ever.  That's kind of revealing because Wesleyan and Western Conn are less than an hour away from each other.  Williams is farther (2.5 hours maybe), but Western Conn came to them, which it might have been willing to do as well if invited. So, it took either an SoS bump from the prior year or a couple of coaches trying to do what is right for Western Conn to get a chance against them.  As it turns out, neither Wesleyan, which Western Conn beat, nor Williams, which beat Western Conn, made it to the NCAA tournament and Western Conn won its AQ and did.

[Kuiper]

SimpleCoach posted this in his own thread, but I wanted to call attention to it for those primarily interested in the NCAA championship selection criteria rule change recommendation.

https://www.youtube.com/watch?v=kMvVzjc6sNs

Very interesting and well worth your time if you want to learn more about the proposal.  Coach Bianco is clearly as well versed about the proposal as you can be and he's tried to get as much access to more information/clarification as he can.

A few quick reactions:

1.  Coach Bianco identifies that the key issue is that the algorithm removes certain games called "bad wins" from consideration.  He mentioned one team that would have been ranked #1 in 2023 would have had 80% of its schedule removed. 

He kept saying the games were removed, but I don't think that's really what he meant and repeating it is misleading.  I think he meant that the wins count in the W-L %, but not in the strength of schedule calculation.  If your team loses to lower ranked team, that "upset" game very much matters. 

So, Bianco's complaint is not that the games are removed from the algorithm altogether, but that he doesn't know which games are removed and which games are kept for SoS purposes.  He specifically mentioned an example in the Landmark Conference where Catholic and Drew both played and beat Susquehanna, but the game was removed (presumably just for SoS purposes) for Drew, but not for Catholic (I think I have that right) in the 2022 or 2023 simulation and he couldn't figure out why.  My guess is that the algorithm's setting for when a win is considered a "bad" win is a function of the gap between the two teams' relative rankings and the gap met the standard in the case of Drew, but not in the case of Catholic (whether measured at the end or when they played each other).     

I'm sympathetic that it should be transparent as to what the gap needs to be to trigger the game's removal for SoS purposes and when that gap is measured.  On the latter point, I assume the algorithm measures these things weekly, but the final measurement is based on the full season.  That means no team is advantaged/disadvantaged by playing a weaker conference team first rather than last.  You certainly don't want timing to be a big sticking point in coming up with conference schedules for instance, but the Championships Committee should confirm that.  On the former point, knowing the numerical or percentage amount of the gap is probably helpful in understanding how low down you can go in scheduling non-conference opponents, but only in a rough way since you wouldn't know either ranking when committing to the schedule.  If only really large gaps get thrown out for SoS purposes, then you might schedule differently than if small or medium-sized gaps get thrown out too.  This is a little bit of a "I want to know how to game the system" complaint, but it's also a "I want to know why one team made it and one team didn't" that goes to the legitimacy of the algorithm and therefore the selections, so it's better to be transparent about it even if it aids in gaming the system (which we all know coaches will do once they understand it).

2.  Coach Bianco asserted that he didn't think this would hurt NESCAC and UAA schools and, in fact, it could hurt regional access because the algorithm, by removing regional committee input etc, could give all the Pool C slots to teams from the Northeast.

He's right that the algorithm would not consider regional access, whereas the national selection committee might.  Since I live in Region X, I can represent the extreme view.  There are only 4 AQs in Region X (3 if the ASC is gone) and many years under the old system only the AQs got bids from Region X.  Maybe 1 or 2 other teams in a good year.  The reason there are so few Pool C bids is because the second place teams typically didn't have high enough SoS and/or had too many "bad" wins.  That's because they historically have had to play double round robins within their conferences due to the lack of any DIII opponents out of conference that don't require flights.  So, the algorithm might make things better and the floor is basically the status quo or close to it.  At least from Region X's perspective, I doubt anyone believes this will hurt them. 

I wasn't satisfied with his explanation about the NESCAC and UAA schools. He just said he thought they would continue to make it because they play strong schedules and win a lot of games.  I don't think anyone disagrees with that for the top of those conferences, but the question is about the middle of the conference.  It would be interesting to know if Williams or Western Connecticut would have made it in 2022 since he apparently has that information.  Also, he mentioned that it would affect seeding even more than selection, which means it could affect hosting.  That alone sometimes can tip the balance in terms of advancing in the tournament.

3.  I still wonder if travel costs are an unstated rationale (or at least added benefit that is making this popular for the deciders)

In some parts of the country, it's a simple decision as to whether you play a strong or weak team as part of your non-conference schedule.  You drive within an hour or so in one direction rather than the other.  And in some conferences, the quality is so high from top to bottom, that you don't really need a SoS bump out of your non-conference schedule, so you just schedule mostly easy nearby opponents. In other parts of the country and other conferences, though, the only way to offset the bad win effect of your conference games is to travel far, usually involving high expenses and missed classes, to play teams with high SoS value.  If the algorithm reduces the "bad win" problem, it reduces the need to travel as much for those schools.  You still need a few strong opponents, but not as many.  In the SCIAC, they switched from a full double round robin to a partial one to open up more slots for non-conference games they could use to increase SoS.  Since there is only one non-conference DIII team within 6-7 hours, that means the SCIAC teams voted to increase their costs to fly to other parts of the country in an effort to get maybe 1 more team in the NCAA tournament every so often. 

Reducing the incentive to travel far distances is nowhere stated in the Championships Committee's memo explaining the rationale for the recommendation and Coach Bianco does not mention it either, so I may be mistaken.  Nevertheless, of the 12 members of the Championships Committee making this recommendation, 7 are Athletic Directors and 2 more are Associate Athletic Directors. They see the big picture as to how much their schools are paying for travel each year.  One of them is the Athletic Director of SUNY New Paltz, which just announced it is initiating it's two year notice to leave the SUNYAC because the travel costs and burden are too high and it is going to go to another conference where they can reduce travel costs.  It has to be at least in the back of her mind and others who might be in the same boat as institutions.

[stlawus]

If this results in the teams most deserving of making tournaments, then it's worth it.  While all this is very complicated, the ultimate goal should be very simple:  Those who are deserving get in.  On the soccer and basketball side of things, the committees over the past several seasons have swung way too far towards selecting the best teams rather than the teams most deserving of making it. There are 2 major examples in both these sports, and one of them has already been mentioned which is Williams/WestConn in 2022.  Williams may have been a better team, but there should be no scenario where a team gets an at-large bid that has more losses and ties than wins.  You can talk about SOS until the cows come home, but West Conn beat Vassar on the road that season.  Vassar was a few penalties in the rain away from advancing to the sweet 16. 

Rochester in men's basketball last year is the other major example. There has never been a hard win% floor in the manual, but Rochester went 16-9 that season. I think only 1 team had ever made the tournament with that record until then. I've already seen the D3 hoops world complaining about this change by citing the former Empire 8 commissioner leading the charge for the new system as a result of Rochester getting that bid over a team like Nazareth who were 21-6. Again, talk about their SOS all you want, but you still have to win games.  It's about who deserves getting in, not who is better.  Rochester lost in the first round that tournament anyways, so the case that they were better fell flat regardless.

Just being in a tough conference should not guarantee you tournament access, that is what the AQ is for.  Of course not all conferences are the same in terms of quality, but the whole point of division III is to strike the balance between equity and quality, not simply favor the leagues and schools that have the advantages in terms of resources.  I have no issue admitting that part of my viewpoint is from an angle of pettiness (mostly on the basketball side of things) in seeing the folks who think they are in "charge" of all things division III upset that now they might actually have to talk about different schools and leagues instead of their annual favorites.

[mngopher]

One good example of this from the 2023 season is Mount Union. 16-2-2 with wins over Calvin, John Carroll, and CWRU. Season ended in a shootout (vs another NCAA tournament team) in their conference tournament. Not only did they not make the NCAA tournament, they weren't even ranked in the final regional rankings. All (I'm guessing) due to a .529 SOS. Way too much focus on the SOS number there. I'd be curious to see what the new system would have done with them.

[Kuiper]

One good example of this from the 2023 season is Mount Union. 16-2-2 with wins over Calvin, John Carroll, and CWRU. Season ended in a shootout (vs another NCAA tournament team) in their conference tournament. Not only did they not make the NCAA tournament, they weren't even ranked in the final regional rankings. All (I'm guessing) due to a .529 SOS. Way too much focus on the SOS number there. I'd be curious to see what the new system would have done with them.

Bianco said in the podcast that Mount Union would have not only made the tournament, but it would have been highly ranked because once you dropped their low SoS wins, their SoS would have gone up greatly because of the opponents you cited, plus teams in their conference.