"Overall, I'm pretty certain it was the right call, but I think your numbers are off. "
The numbers are open to a little bit of debate. You need to start with the league baselines and make a number of adjustments for the specific situation. My point wasn't to define the exact probability down to the third decimal point. It was to define the framework and point people in the direction of the right numbers.
It was also to emphasize just how wrong the status quo is. The thing about the NFL is that everybody thinks that punting is the right thing to do because everyone else is doing it. Its like a classroom where nobody did the homework. They all just copy the other's kids' homework.
By this point even the pro-punt crowd has gone from "that was obviously the dumbest decision ever" to well if you assign this probability to this component event, its a close call.....
"There's only a 15% difference in Indy scoring a td from their own 30 yard line compared to new englands 29??"
After thinking about this for a week, I think it might be closer to 20% difference in WP. Still less than you thought, right? Thats one of the things about looking up the data. It sometimes different than you thought it would be. Your memory can play tricks with you. I looked at these numbers a whole bunch of different ways. Only Colts drives. All NFL drives. Only drives in the last 2 minutes, etc. etc. They all came back in the same 15-20% difference range.
I'd also emphasize that's the difference in win prob. A diff between 50 and 30 WP still means you score touchdowns 67% more often with the ball at the 28 (50/30 = 1.6666)
"I bet the true % of td's in this situation with timeouts, and 2:00 left is probably closer to 10%."
There is a graph of all of the times teams have been in this position in the final 2 minutes of a game here:
https://www.advancednflstats.com/2009/11/two-minute-drill.html
As you can see, its far higher than 10% from your own 30. With the other team in a prevent defense and your team willing to use all four downs, you score the TD more than you might think.
"2 pt conversion pass plays on the road have hit at a 34.8% rate the last 3 years. NE is a considerably better passing team than most teams, so I would guess the chances they got the 4th and 2 were closer to 50%. "
There's a number of reasons why I think its higher than 50%, but 50% is in the acceptable "open to debate" range. As I said before, this isn't designed to say there is a 53.47% chance of success. Any number for the probability in this particular situation is going to be an estimate. The point of this whole exercise is that estimate should have some basis in reality.
BTW...why exactly coaches always seem to pass on 2 pt plays when runs typically work better is another good topic for debate.
You should realize that those 2 pt conversion numbers are slightly skewed. If the holder bobbles the extra point snap and then stands up to make a desperation throw to the endzone, the NFL counts it as a failed 2 pt try. In reality, its a failed 1 pt try.