Does that mean logisitic regression is broken? Of course not. Maximum likelihood theory tells us that the estimates converge in distribution, not that the mean converges. Side note: I think this would make a great example for a first year graduate course in probability.
A commentor asked whether logistic regression is biased if we ignore the case of perfect separation. I decided to examine this using simulation. Another question I had is whether the estimated probabilities were biased. Using Rmarkdown, I walked through the exploration. Here goes!
We start with some utility functions.
To examine the small sample bias, we will simulate a large number of data sets, compute the logistic regression fit and extract the coefficients and estimated probabilities.
The probability of getting perfect separation in these simulations is so small that even in this large simulation (10,000 sample fits), we still don't observe a single perfect separation. So this esstentially ignores the issue of perfect separation.
perfect separation was never observed. We can examine this a little more formally by examining the mean,
median and corresponding standard errors.
Question 2: Are the fitted probabilities biased?